[ci] build the PPU image from this repo - #613
Open
tiankongdeguiji wants to merge 4 commits into
Open
Conversation
The PPU image was built out of tree, so it drifted from requirements/ whenever the release pins moved and could not be reproduced from a checkout. Move the build in: docker/Dockerfile.ppu installs the PPU torch/triton/faiss builds from the FlyTiger Eco pypi index on top of the vendor PPU base image and then installs requirements.txt like the other images do. PPU build and promotion get their own scripts because the image is validated on its own cadence and may lag the cpu/cu* images. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5JspSJLDuxSnprexTKwba
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5JspSJLDuxSnprexTKwba
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H5JspSJLDuxSnprexTKwba
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Build the PPU image from this repo instead of an out-of-tree addons checkout, and pick up the PPU artifacts' new home.
The PPU image was built outside the repo, so it drifted from
requirements/whenever the release pins moved and could not be reproduced from a checkout.docker/Dockerfile.ppunow installs the PPU torch/triton/faiss/fbgemm builds from the vendor pypi index on top of the vendor PPU base image, then installsrequirements.txtlike the other images do. Thetorch==2.11.0/fbgemm-gpu==1.6.0/torchrec==1.6.0pins inrequirements/runtime.txtare satisfied by the PPU builds (their local version+v0.1.0.ppu2.1.1matches the pin), so no non-PPU wheel is pulled in.Changes:
docker/Dockerfile.ppu: base imageppu:v2.1.1-cuda13.0-ubuntu24-py312, PPU wheels from the vendor index,mirrors.aliyun.comfor everything else. Drops thePIP_INDEX_URL+sitecustomize.py/aiextpypi.pthSTS shim the old out-of-tree Dockerfile carried; it is no longer needed.scripts/build_docker_ppu.sh/scripts/promote_docker_ppu.sh: build and promotion for the single PPU tag. Kept separate frombuild_docker.sh/promote_docker.shbecause the PPU image is built and validated on its own cadence and may lag the cpu/cu* images; it also never takes the<tag>/latestaliases..github/workflows/unittest_ppu_ci.yml: run the PPU lane on the freshly built image (switched back totzrec-devel:1.2-ppuafter promotion).docs/source/develop.md: document the PPU image build.Test Plan
torch 2.11.0+v0.1.0.ppu2.1.1,triton 3.6.0+v0.2.0.ppu2.1.1,faiss 1.14.1+v0.1.0.ppu2.1.1,fbgemm_gpu 1.6.0+v0.1.0.ppu2.1.1,torchrec 1.6.0;torch,triton,faiss,fbgemm_gpu,torchrec,torchmetrics,graphlearn,pyfg,tensorboard,sklearnall import.pip install -r requirements.txtreports torch/fbgemm-gpu/torchrec as already satisfied, i.e. the shared requirements files do not re-resolve a non-PPU torch.--no-cache): an earlier build had cached a layer from when the vendor index still proxied the upstreamfbgemm_gpuwheel, which loads against real CUDA but not the PPU CUDA shim (undefined symbol: __cudaGetKernel, version libcudart.so.13). The vendor index now serves only PPU artifacts for that name, and the cold build resolvesfbgemm_gpu-1.6.0+v0.1.0.ppu2.1.1.scripts/ci/ci_test_ppu.sh, fulltzrec/tests/run.py) on the built image.