chore: add publication for linux/arm64 for -cpu containers

[vlsi]

Can you please start releasing arm64 images as well?

Related Issue

Closes #616

Checklist

• I have read the CONTRIBUTING guidelines.
• I have added tests to cover my changes.
• I have updated the documentation (docs folder) accordingly.

[gemini-code-assist]

Note

Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported.

[greptile-apps]

Greptile Summary

This PR adds linux/arm64 to the CPU container build targets by updating the release workflow platform list and guarding the onnxruntime-openvino pip install (which only ships x86_64 wheels) behind a uname -m architecture check.

• release.yaml: Adds linux/arm64 to the platforms field for the -cpu job; QEMU and Docker Buildx are already set up in the shared release_docker_container.yaml workflow, so the infrastructure change is minimal.
• Docker.template.yaml + Dockerfile.cpu_auto: All three Dockerfile stages (builder no-root, builder with-root, testing) now conditionally skip onnxruntime-openvino on non-x86_64 hosts. The template's {{main_install}} block propagates the guard correctly to all three generated stages via Jinja's string-replace filters.

Confidence Score: 5/5

Safe to merge — the previously identified onnxruntime-openvino incompatibility is properly addressed in all three Dockerfile stages.

The only known blocker for arm64 support was the unconditional onnxruntime-openvino install, which only ships x86_64 wheels. All three install sites in Dockerfile.cpu_auto are now guarded by uname -m, the template in Docker.template.yaml is updated consistently, and the Jinja rendering ensures the guard propagates correctly to every stage. The QEMU and Buildx infrastructure was already in place in the shared workflow.

No files require special attention.

Important Files Changed

Filename	Overview
.github/workflows/release.yaml	Added linux/arm64 to the CPU container build platform list; underlying QEMU/Buildx setup already exists in the shared workflow.
libs/infinity_emb/Docker.template.yaml	Added uname -m arch guard for onnxruntime-openvino in the main_install block; since this block is Jinja-interpolated into all three Dockerfile stages, the guard propagates correctly everywhere.
libs/infinity_emb/Dockerfile.cpu_auto	All three onnxruntime-openvino install lines now skip on non-x86_64; consistent with the updated template and Jinja rendering.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[release.yaml\ndocker-container-push-cpu] -->|platforms: linux/amd64,linux/arm64| B[release_docker_container.yaml\nBuildx + QEMU]
    B --> C[Dockerfile.cpu_auto\nFROM ubuntu:22.04 AS builder]
    C --> D{uname -m}
    D -->|x86_64| E[pip install onnxruntime-openvino]
    D -->|aarch64| F[skip — no arm64 wheel]
    E --> G[COPY infinity_emb\ninstall with root]
    F --> G
    G --> H{uname -m}
    H -->|x86_64| I[pip install onnxruntime-openvino]
    H -->|aarch64| J[skip]
    I --> K[FROM builder AS testing]
    J --> K
    K --> L{uname -m}
    L -->|x86_64| M[pip install onnxruntime-openvino]
    L -->|aarch64| N[skip]
    M --> O{TARGETPLATFORM}
    N --> O
    O -->|linux/arm64| P[pytest test_api_with_dummymodel.py]
    O -->|linux/amd64| Q[pytest test_api + test_sentence_transformers]
    P --> R[FROM tested-builder AS production]
    Q --> R

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[release.yaml\ndocker-container-push-cpu] -->|platforms: linux/amd64,linux/arm64| B[release_docker_container.yaml\nBuildx + QEMU]
    B --> C[Dockerfile.cpu_auto\nFROM ubuntu:22.04 AS builder]
    C --> D{uname -m}
    D -->|x86_64| E[pip install onnxruntime-openvino]
    D -->|aarch64| F[skip — no arm64 wheel]
    E --> G[COPY infinity_emb\ninstall with root]
    F --> G
    G --> H{uname -m}
    H -->|x86_64| I[pip install onnxruntime-openvino]
    H -->|aarch64| J[skip]
    I --> K[FROM builder AS testing]
    J --> K
    K --> L{uname -m}
    L -->|x86_64| M[pip install onnxruntime-openvino]
    L -->|aarch64| N[skip]
    M --> O{TARGETPLATFORM}
    N --> O
    O -->|linux/arm64| P[pytest test_api_with_dummymodel.py]
    O -->|linux/amd64| Q[pytest test_api + test_sentence_transformers]
    P --> R[FROM tested-builder AS production]
    Q --> R

Loading

_{Reviews (2): Last reviewed commit: "chore: add publication for linux/arm64 f..." | Re-trigger Greptile}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 68160b7e4d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[michaelfeil]

@wirthual can you review

[vlsi]

For reference:

• this PR makes arm64 container workable for my Apple M1
• rerank with container-based arm64 build, BAAI/bge-reranker-v2-m3 model is like 10 times slower than the same rerank when infinity executes with uv (no Docker)

The difference comes from the fact that Docker uses Linux torch libraries while local execution via uv goes with Apple Silicon optimized libraries.

So I ended up with using uv ... to start infinity rather than docker image.

The difference might be worth documenting so others don't bump into it.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.00%. Comparing base (1eb4396) to head (686b0b3).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #665      +/-   ##
==========================================
- Coverage   80.05%   80.00%   -0.06%     
==========================================
  Files          43       43              
  Lines        3485     3485              
==========================================
- Hits         2790     2788       -2     
- Misses        695      697       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[wirthual]

Hi,

thanks for your contribution and sharing your findings.

How do you benchmark the rerankers? Can you share the commands you use to run the servers? Also can you dump the installed packages in your environments? That would greatly help to look deeper into it.