Automate SBOM generation for container images - #805
Conversation
jayavenkatesh19
added
feature request
jameslamb
left a comment
jameslamb
left a comment
There was a problem hiding this comment.
Thanks for getting this started! Doing it in a multi-stage build is a really cool idea, nice way to make it work with the third-party actions we use to build and publish images (no need to manually invoke docker build ourselves).
I see it's been a few weeks since the last activity here... do you need help with anything? Or is this effort just paused right now?
|
Hey @jameslamb! This effort has been on pause as I was focusing on creating the notebook for my talk for PyData Boston. I also have PRs open on ci-imgs and devcontainers up for a while. It'd be great if you could glance over them and leave your review for these! I've also looked at the SBOMs generated and there's a lot of fields pertaining to individual files in the Docker images. I am not sure if we need to include every file in the image or if a list of libraries included with the image would do. The current |
|
Ok thanks for the update, sorry for the very long delayed response! Doing this in a multi-stage docker build is a nice approach, I support it.
The more we can include and confidently attribute to a source + a set of licenses, the better. Will just have to see how scanning tools handle the SBOMs we're generating. |
jameslamb
left a comment
jameslamb
left a comment
There was a problem hiding this comment.
Left a few suggestions, things I noticed after reviewing similar changes in rapidsai/ci-imgs#309
|
|
||
| RUN --mount=type=bind,from=notebooks-build,source=/,target=/rootfs,ro \ | ||
| mkdir -p /out && \ | ||
| syft scan \ |
There was a problem hiding this comment.
Similar to rapidsai/ci-imgs#309 (comment), I think this should be moved to a mounted-in script so we only have to set the syft configuration in 1 place.
| ARG LINUX_VER=${LINUX_DISTRO}${LINUX_DISTRO_VER} | ||
|
|
||
| ARG RAPIDS_VER=26.02 | ||
| ARG SYFT_VER=1.32.0 |
There was a problem hiding this comment.
Similar to rapidsai/ci-imgs#309 (comment), could you please move the syft version and Alpine tag into versions.yaml?
To be fair, there wasn't a versions.yaml when you started this PR, it was only very recently added: #834
…ingle stage. Signed-off-by: Jaya Venkatesh <jjayabaskar@nvidia.com>
0776571 to
6a46959
Compare
Feedback from James is that the purpose of this PR was not clear, and that the summary was pretty heavy on jargon. The jargon did not phase me, presumably because I've been swimming in this stuff for longer/deeper. The purpose of this PR is to surface image contents in better ways. As part of our release process, the images that this repo produces get scanned with various tools to find CVEs and malware. These tools are prone to misidentifying some components, and that has been especially true for conda packages historically. We have frequently needed to override the detected package lists/SBOM via JIRA tickets with our security tools team. Part of filing those tickets is providing them with the actual contents of the images, which we often have to discover by looking into build logs. That's where this PR comes in. Instead of looking at build logs, we should export our known environment information to a more readily usable location. In theory, this could replace the scans, but I think it's better to keep doing the scans and use our manifests as validation/correction where necessary. That keeps us in line with how security tools team manages other projects. **Posted by Codex (GPT-5.6) on behalf of Michael Sarahan. Treat this description as LLM-generated.** ## Summary Add a signed, OCI-referrer RAPIDS image provenance manifest for published `base` and `notebooks` image digests. - Export exact `conda-meta` package facts through minimal BuildKit targets rather than downloading image layers. - Attach a platform-specific manifest to every pushed architecture image, then attach a multiarch index manifest that links those platform records. - Record the image digest, platform, source SHA, workflow URL, build arguments, and conda package name/version/build/channel/source URL. - Keylessly sign each attached OCI artifact with GitHub Actions OIDC and Cosign. - Keep pURLs explicitly unmapped until a verified upstream mapping is available; this intentionally does not invent `pkg:conda/...` identifiers. ## Relationship to #805 #805 is complementary. It generates and embeds a CycloneDX SBOM in the final image. This change makes digest-bound build facts discoverable as registry metadata and supplies the platform-to-multiarch relationship needed by scan triage without pulling image layers. If #805 lands, the preferred follow-up is to add its SBOM digest/reference to this manifest rather than maintain a separate provenance model. Its Dockerfile stage renaming will require a small conflict resolution in this branch's `provenance-base` and `provenance-notebooks` export targets.
2 participants
Towards https://github.com/rapidsai/build-infra/issues/280
Adds SBOM to
baseandnotebookimagessyft-basewith the Syft binary installed on a minimal alpine 3.20 image.base-buildandnotebooks-buildto differentiate it from the final image.base-sbom/notebooks-sbomwhere the built stage is mounted to a specified location on thesyft-basestagesyft-scanis done on the mounted location, and an SBOM is generated.