Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,18 @@ jobs:
run: |
sh .github/validate-vars.sh --check
sh docker/gpg-gate-check.sh
- uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
# Layers cached in Actions, one scope per variant; release reads the same scope, so a
# tag rebuilds only what the merge changed.
- name: Build the image the example names
run: docker build -t ghcr.io/katoptra/toolbox:${{ matrix.variant }}-dev -f docker/${{ matrix.variant }}.Dockerfile .
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
with:
context: .
file: docker/${{ matrix.variant }}.Dockerfile
load: true
tags: ghcr.io/katoptra/toolbox:${{ matrix.variant }}-dev
cache-from: type=gha,scope=${{ matrix.variant }}
cache-to: type=gha,scope=${{ matrix.variant }},mode=max
- name: The example's tools resolve inside the image
working-directory: examples/${{ matrix.variant }}
run: task run -- task tools
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ jobs:
file: docker/${{ matrix.variant }}.Dockerfile
platforms: linux/amd64,linux/arm64
push: true
cache-from: type=gha,scope=${{ matrix.variant }}
cache-to: type=gha,scope=${{ matrix.variant }},mode=max
tags: |
${{ steps.tags.outputs.semver }}
${{ steps.tags.outputs.major }}
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -582,7 +582,7 @@ never touch Docker Hub.

| Variant | Base | Tools | For |
|---|---|---|---|
| `rsync` | ubuntu 24.04 | rsync, gnupg, xz, curl, perl (shasum), go-task, AWS CLI v2 trimmed to s3 and sts | rsync upstreams into a bucket: ctan, tlnet |
| `rsync` | ubuntu 24.04 | rsync, gpgv, xz, curl, perl (shasum), go-task, AWS CLI v2 trimmed to s3 and sts | rsync upstreams into a bucket: ctan, tlnet |
| `proton` | python 3.13 slim | proton-drive, age, git, go-task, boto3, requests, pytest, ruff, and `s3`, a boto3 get/put | Proton Drive sinks: github through the proton engine, dropbox through its own Python |

Both bind-mount the repository at `/work`, set `TASK_REMOTE_OFFLINE=1` and
Expand Down
13 changes: 12 additions & 1 deletion docker/proton.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,20 @@ COPY --from=fetch /usr/local/bin/proton-drive /usr/local/bin/age /usr/local/bin/
COPY toolchain.lock.toml /etc/toolchain.lock.toml
COPY docker/s3.py /usr/local/bin/s3

# ponytail: botocore ships 432 service models and s3 calls one, so all but s3, sts and
# the data-root *.json stay behind. Ceiling: a boto3 client for any other service dies on
# a model lookup; add it to the keep list. pip leaves once the pins are in: a runtime
# image has no business installing anything.
RUN python - <<'PY'
import subprocess, tomllib
import pathlib, shutil, subprocess, tomllib
lock = tomllib.load(open("/etc/toolchain.lock.toml", "rb"))
pins = [f"{k}=={v}" for t in ("packages", "test_packages") for k, v in lock["python"][t].items()]
subprocess.check_call(["pip", "install", "--no-cache-dir", "--no-deps", *pins])
subprocess.check_call(["pip", "uninstall", "-y", "pip"])
import botocore
for p in (pathlib.Path(botocore.__file__).parent / "data").iterdir():
if p.is_dir() and p.name not in ("s3", "sts"):
shutil.rmtree(p)
PY

RUN python - <<'PY'
Expand All @@ -53,6 +62,8 @@ assert lock["age"]["version"] in first(["age", "--version"])
assert lock["task"]["version"] in subprocess.check_output(["task", "--version"], text=True)
assert first(["git", "--version"]).startswith("git version 2.")
assert subprocess.run(["s3"], capture_output=True).returncode == 2 # usage; boto3 imports
assert subprocess.run(["python", "-m", "pip"], capture_output=True).returncode == 1
import boto3; boto3.client("s3", region_name="auto", aws_access_key_id="x", aws_secret_access_key="x")
PY

# Inside a run there is no network for Taskfiles: the mirror's .task/remote cache rides
Expand Down
5 changes: 3 additions & 2 deletions docker/rsync.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,10 @@ RUN set -eu; \

FROM ubuntu:24.04@sha256:33ceb71981b602c1a7443a53469e4dba065f7503eab3078a2d7a57a2ab987517 AS toolbox

# perl carries shasum. Each RUN deletes its own scratch: a layer keeps what it leaves.
# perl carries shasum; gpgv alone, since the engine verifies and never signs. Each RUN
# deletes its own scratch: a layer keeps what it leaves.
RUN apt-get update && apt-get install -y --no-install-recommends \
rsync gnupg xz-utils curl ca-certificates perl python3 \
rsync gpgv xz-utils curl ca-certificates perl python3 \
&& rm -rf /var/lib/apt/lists/*
COPY --from=fetch /usr/local/bin/task /usr/local/bin/task
COPY --from=fetch /usr/local/aws-cli /usr/local/aws-cli
Expand Down