feat(plugin-oci): oci_index groups per-platform images into one - #383
Open
raphaelvigee wants to merge 1 commit into
Open
feat(plugin-oci): oci_index groups per-platform images into one#383raphaelvigee wants to merge 1 commit into
raphaelvigee wants to merge 1 commit into
Conversation
`docker_build(platforms = [...])` is one buildx invocation: one Dockerfile,
one context, one set of build args, one stage, for every platform at once.
`context_by_platform` lets the deps differ and `TARGETPLATFORM` lets the
Dockerfile branch, but it is still one recipe — so platforms that need
genuinely different ones (a different base, a different package manager, a
stage that exists on one arch only) have nowhere to put the difference.
`oci_index` takes images built separately and makes them one image:
docker_build(name = "amd64", dockerfile = ":Dockerfile.amd64",
platforms = ["linux/amd64"], context = [...])
docker_build(name = "arm64", dockerfile = ":Dockerfile.arm64",
platforms = ["linux/arm64"], context = [...])
oci_index(name = "img", images = [":amd64", ":arm64"])
`//pkg:img` is then one image everywhere downstream — `oci_push` sends the
whole manifest list under one tag, `oci_load` picks the host's instance,
`bases` resolves the right platform out of it. The split stays a build-time
detail rather than something the rest of the repo has to know about.
A separate rule rather than a `docker_build` mode: a single-Dockerfile
multi-platform build and an N-Dockerfile one would otherwise be one rule in
two modes with half the attributes inert in each. Grouping is also not a
build — nothing is compiled, executed, or handed to a daemon. It takes any
layout-producing target, not only `docker_build`; mixing a hand-assembled
`oci_image` for one arch with a Dockerfile build for another is exactly the
case that has nowhere else to go.
Three things worth calling out:
- The platform comes from the image **config** when the index entry carries
no annotation, which is the normal case for a single-platform
`--output type=oci` build — so that is the path that runs most of the
time, not a fallback. `architecture` and `os` are required config fields,
and the config is what a runtime actually reads. The variant is carried
through: `linux/arm/v7` and `linux/arm/v6` are different machines.
- Two inputs claiming one platform is a hard error naming both targets.
Keeping one would make which image ships depend on the order `images`
happens to be written in.
- `images` is pinned to the archive output group. Unpinned, the dep also
stages the sibling `digest` group's text file and the layout root can no
longer be found by its `oci-layout` marker among two unrelated files —
the same reason `oci_push` and `oci_load` pin theirs.
Blobs are carried by reference, so grouping N images copies no layer bytes
through memory, and a base layer shared between platforms is stored once.
Nothing is rebuilt: each input keeps its own digest and caches on its own
inputs, so changing the arm64 Dockerfile does not rebuild amd64.
The def hash carries the ordered, normalized `images` addresses — `hashin`
folds a sorted, unlabeled multiset of dep hashouts with no address and no
output-group selector, and the manifest list's entry order is part of its
bytes.
`base_layout_path` moves to `mod.rs` as `layout_path` since `oci_image` and
`oci_index` now both need it; it takes the attribute name so the error says
`base` or `images` rather than always the former.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WqQZxsTqanJRWLWdPSchBo
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Top of stack #379: #159 → #378 → #381 → this.
The gap
docker_build(platforms = [...])is one buildx invocation. One Dockerfile, one context, one set of build args, one--targetstage, for every platform at once.context_by_platformlets the deps differ andTARGETPLATFORMlets the Dockerfile branch — but it is still one recipe. Platforms needing genuinely different ones (a different base, a different package manager, a stage that exists on one arch only) have nowhere to put the difference.The rule
//pkg:imgis then one image everywhere downstream —oci_pushsends the whole manifest list under one tag,oci_loadpicks the host's instance,basesresolves the right platform out of it. The split stays a build-time detail instead of leaking into what the repo refers to.Why a separate rule
As a
docker_buildmode, a single-Dockerfile multi-platform build and an N-Dockerfile one would be one rule in two modes with half the attributes inert in each — the shapecompatibilityandproduct-visionalready rejected foroci_image. Grouping is also not a build: nothing is compiled, executed, or handed to a daemon.It accepts any layout-producing target, not only
docker_build. Mixing a hand-assembledoci_imagefor one arch with a Dockerfile build for another is exactly the case with nowhere else to go.Three details
--output type=ocibuild, so it is the path that runs most of the time rather than a fallback.architecture/osare required config fields and the config is what a runtime reads. The variant is carried through:linux/arm/v7andlinux/arm/v6are different machines.imageshappens to be written in.imagesis pinned to the archive group. Unpinned, the dep also stages the siblingdigestgroup's text file, and the layout root can no longer be found by itsoci-layoutmarker among two unrelated files. Caught by the e2e; same reasonoci_push/oci_loadpin theirs.Cost
Blobs are carried by reference — with #378's located-not-loaded blobs, grouping N images copies no layer bytes through memory, and a layer shared between platforms is stored once. Nothing is rebuilt: each input keeps its own digest and caches on its own inputs, so changing the arm64 Dockerfile does not rebuild amd64.
The def hash carries the ordered normalized
imagesaddresses —hashinfolds a sorted, unlabeled multiset of dep hashouts with no address and no output-group selector, and the manifest list's entry order is part of its bytes.Tests
6 unit + 3 engine e2e, none docker-gated: the e2e builds two
oci_imagetargets with different layers, entrypoints and env — something onedocker_buildcould not produce — groups them, and asserts the entry point is a manifest list naming both platforms with each entry still pointing at the image its own target built. Plus the duplicate-platform error naming both targets, and blob dedup across platforms.base_layout_pathmoves tomod.rsaslayout_pathsince two drivers need it now; it takes the attribute name so the error saysimagesrather than alwaysbase.🤖 Generated with Claude Code
https://claude.ai/code/session_01WqQZxsTqanJRWLWdPSchBo