Skip to content

feat(plugin-oci): oci_index groups per-platform images into one - #383

Open
raphaelvigee wants to merge 1 commit into
feat/oci-load-auto-tagfrom
feat/oci-index
Open

feat(plugin-oci): oci_index groups per-platform images into one#383
raphaelvigee wants to merge 1 commit into
feat/oci-load-auto-tagfrom
feat/oci-index

Conversation

@raphaelvigee

Copy link
Copy Markdown
Member

Top of stack #379: #159#378#381 → this.

The gap

docker_build(platforms = [...]) is one buildx invocation. One Dockerfile, one context, one set of build args, one --target stage, for every platform at once. context_by_platform lets the deps differ and TARGETPLATFORM lets the Dockerfile branch — but it is still one recipe. Platforms needing genuinely different ones (a different base, a different package manager, a stage that exists on one arch only) have nowhere to put the difference.

The rule

docker_build(name = "amd64", dockerfile = ":Dockerfile.amd64",
             platforms = ["linux/amd64"], context = [...])
docker_build(name = "arm64", dockerfile = ":Dockerfile.arm64",
             platforms = ["linux/arm64"], context = [...])

oci_index(name = "img", images = [":amd64", ":arm64"])

//pkg:img is then one image everywhere downstream — oci_push sends the whole manifest list under one tag, oci_load picks the host's instance, bases resolves the right platform out of it. The split stays a build-time detail instead of leaking into what the repo refers to.

Why a separate rule

As a docker_build mode, a single-Dockerfile multi-platform build and an N-Dockerfile one would be one rule in two modes with half the attributes inert in each — the shape compatibility and product-vision already rejected for oci_image. Grouping is also not a build: nothing is compiled, executed, or handed to a daemon.

It accepts any layout-producing target, not only docker_build. Mixing a hand-assembled oci_image for one arch with a Dockerfile build for another is exactly the case with nowhere else to go.

Three details

  • The platform comes from the image config when the index entry has no annotation — which is the normal case for a single-platform --output type=oci build, so it is the path that runs most of the time rather than a fallback. architecture/os are required config fields and the config is what a runtime reads. The variant is carried through: linux/arm/v7 and linux/arm/v6 are different machines.
  • Two inputs claiming one platform is a hard error naming both targets. Keeping one would make which image ships depend on the order images happens to be written in.
  • images is pinned to the archive group. Unpinned, the dep also stages the sibling digest group's text file, and the layout root can no longer be found by its oci-layout marker among two unrelated files. Caught by the e2e; same reason oci_push/oci_load pin theirs.

Cost

Blobs are carried by reference — with #378's located-not-loaded blobs, grouping N images copies no layer bytes through memory, and a layer shared between platforms is stored once. Nothing is rebuilt: each input keeps its own digest and caches on its own inputs, so changing the arm64 Dockerfile does not rebuild amd64.

The def hash carries the ordered normalized images addresses — hashin folds a sorted, unlabeled multiset of dep hashouts with no address and no output-group selector, and the manifest list's entry order is part of its bytes.

Tests

6 unit + 3 engine e2e, none docker-gated: the e2e builds two oci_image targets with different layers, entrypoints and env — something one docker_build could not produce — groups them, and asserts the entry point is a manifest list naming both platforms with each entry still pointing at the image its own target built. Plus the duplicate-platform error naming both targets, and blob dedup across platforms.

base_layout_path moves to mod.rs as layout_path since two drivers need it now; it takes the attribute name so the error says images rather than always base.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WqQZxsTqanJRWLWdPSchBo

`docker_build(platforms = [...])` is one buildx invocation: one Dockerfile,
one context, one set of build args, one stage, for every platform at once.
`context_by_platform` lets the deps differ and `TARGETPLATFORM` lets the
Dockerfile branch, but it is still one recipe — so platforms that need
genuinely different ones (a different base, a different package manager, a
stage that exists on one arch only) have nowhere to put the difference.

`oci_index` takes images built separately and makes them one image:

    docker_build(name = "amd64", dockerfile = ":Dockerfile.amd64",
                 platforms = ["linux/amd64"], context = [...])
    docker_build(name = "arm64", dockerfile = ":Dockerfile.arm64",
                 platforms = ["linux/arm64"], context = [...])

    oci_index(name = "img", images = [":amd64", ":arm64"])

`//pkg:img` is then one image everywhere downstream — `oci_push` sends the
whole manifest list under one tag, `oci_load` picks the host's instance,
`bases` resolves the right platform out of it. The split stays a build-time
detail rather than something the rest of the repo has to know about.

A separate rule rather than a `docker_build` mode: a single-Dockerfile
multi-platform build and an N-Dockerfile one would otherwise be one rule in
two modes with half the attributes inert in each. Grouping is also not a
build — nothing is compiled, executed, or handed to a daemon. It takes any
layout-producing target, not only `docker_build`; mixing a hand-assembled
`oci_image` for one arch with a Dockerfile build for another is exactly the
case that has nowhere else to go.

Three things worth calling out:

- The platform comes from the image **config** when the index entry carries
  no annotation, which is the normal case for a single-platform
  `--output type=oci` build — so that is the path that runs most of the
  time, not a fallback. `architecture` and `os` are required config fields,
  and the config is what a runtime actually reads. The variant is carried
  through: `linux/arm/v7` and `linux/arm/v6` are different machines.
- Two inputs claiming one platform is a hard error naming both targets.
  Keeping one would make which image ships depend on the order `images`
  happens to be written in.
- `images` is pinned to the archive output group. Unpinned, the dep also
  stages the sibling `digest` group's text file and the layout root can no
  longer be found by its `oci-layout` marker among two unrelated files —
  the same reason `oci_push` and `oci_load` pin theirs.

Blobs are carried by reference, so grouping N images copies no layer bytes
through memory, and a base layer shared between platforms is stored once.
Nothing is rebuilt: each input keeps its own digest and caches on its own
inputs, so changing the arm64 Dockerfile does not rebuild amd64.

The def hash carries the ordered, normalized `images` addresses — `hashin`
folds a sorted, unlabeled multiset of dep hashouts with no address and no
output-group selector, and the manifest list's entry order is part of its
bytes.

`base_layout_path` moves to `mod.rs` as `layout_path` since `oci_image` and
`oci_index` now both need it; it takes the attribute name so the error says
`base` or `images` rather than always the former.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WqQZxsTqanJRWLWdPSchBo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant