Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 96 additions & 2 deletions .github/workflows/anneal.yml
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,7 @@ jobs:
duration=$((end - start))
echo "Docker Pull Time: $duration seconds"
echo "[{\"name\": \"Docker Pull Time\", \"unit\": \"seconds\", \"value\": $duration}]" > pull_time.json

Comment thread
joshlf marked this conversation as resolved.
docker tag ghcr.io/google/zerocopy/anneal:${STEPS_DOCKER_TAG_OUTPUTS_TAG} anneal-ci:local
env:
STEPS_DOCKER_TAG_OUTPUTS_TAG: ${{ steps.docker_tag.outputs.tag }}
Expand Down Expand Up @@ -471,6 +471,100 @@ jobs:
fi
fi

v2_nix_cache:
name: Warm Nix Cache for V2
runs-on: ubuntu-latest
needs: build_docker_env
permissions:
contents: read
id-token: write # Required to exchange GitHub OIDC tokens for Determinate Systems Cache API access
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false

- name: Install Nix
uses: DeterminateSystems/determinate-nix-action@441b9e401ac050c38a07d8313748c5c2d17e8aff # v3.6.1

- name: Run Magic Nix Cache
uses: DeterminateSystems/magic-nix-cache-action@908b263ff629f4cc17666315b7fd3ec127c6244d # v14

# On Ubuntu 24.04 (currently `ubuntu-latest`), AppArmor restricts unprivileged user namespaces by default.
# The Nix build sandbox runs `steam-run` (which uses `bubblewrap`/`bwrap`) during the `mathlib-cache-download`
# phase to create an FHS environment. `bwrap` requires creating a user namespace to set up uid mappings,
# which fails with "Permission denied" unless this restriction is temporarily disabled on the host.
#
# We temporarily disable it right before the `nix build` step and re-enable it immediately after
# to maintain the principle of least privilege.
#
# FIXME(#3412): Deduplicate this with what's repeated below?
- name: Enable unprivileged user namespaces (Ubuntu 24.04)
Comment thread
mdittmer marked this conversation as resolved.
run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0

- name: Warm Nix Cache
run: nix build .#omnibus-archive
working-directory: anneal/v2

# Re-enable the AppArmor namespace restriction to restore the runner host's default security posture.
# `if: always()` ensures this cleanup step runs even if the Nix build fails.
- name: Restore AppArmor restriction
if: always()
run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=1

v2:
name: Run V2 tests
runs-on: ubuntu-latest
# Depending on `v2_nix_cache` avoids duplicate work and ensure `nix build ...` step for this job is fast.
needs: [build_docker_env, v2_nix_cache]
permissions:
contents: read
id-token: write # Required to exchange GitHub OIDC tokens for Determinate Systems Cache API access
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false

- name: Install Nix
uses: DeterminateSystems/determinate-nix-action@441b9e401ac050c38a07d8313748c5c2d17e8aff # v3.6.1

- name: Run Magic Nix Cache
uses: DeterminateSystems/magic-nix-cache-action@908b263ff629f4cc17666315b7fd3ec127c6244d # v14

# On Ubuntu 24.04 (currently `ubuntu-latest`), AppArmor restricts unprivileged user namespaces by default.
# The Nix build sandbox runs `steam-run` (which uses `bubblewrap`/`bwrap`) during the `mathlib-cache-download`
# phase to create an FHS environment. `bwrap` requires creating a user namespace to set up uid mappings,
# which fails with "Permission denied" unless this restriction is temporarily disabled on the host.
#
# We temporarily disable it right before the `nix build` step and re-enable it immediately after
# to maintain the principle of least privilege.
#
# FIXME(#3412): Deduplicate this with what's repeated above?
- name: Enable unprivileged user namespaces (Ubuntu 24.04)
run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0

- name: Build outside-cargo dependencies (cached)
run: |
mkdir -p target
nix build .#omnibus-archive --out-link target/anneal-exocrate.tar.zst
working-directory: anneal/v2

# Re-enable the AppArmor namespace restriction to restore the runner host's default security posture.
# `if: always()` ensures this cleanup step runs even if the Nix build fails.
- name: Restore AppArmor restriction
if: always()
run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=1

# FIXME: Pin this nightly to the same Rust date encoded in
# anneal/v2/flake.nix, or derive it from the archive metadata, so v2 CI is
# reproducible instead of following whatever nightly happens to be latest.
- name: Install latest nightly Rust
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # zizmor: ignore[superfluous-actions]
with:
toolchain: nightly

- name: Run V2 tests
run: cargo test --workspace --all-features # include, e.g., tests that assume exocrate prebuilt
working-directory: anneal/v2

# Used to signal to branch protections that all other jobs have succeeded.
all-jobs-succeed:
Expand All @@ -485,7 +579,7 @@ jobs:
# https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/troubleshooting-required-status-checks#handling-skipped-but-required-checks
if: failure()
runs-on: ubuntu-latest
needs: [build_docker_env, anneal_tests, verify_examples, measure_image_size]
needs: [build_docker_env, anneal_tests, verify_examples, measure_image_size, v2_nix_cache, v2]
steps:
- name: Mark the job as failed
run: exit 1
Loading
Loading