Backport #23646, #23732, #23714: Fix cudf-java native load - #23766
Draft
paul-aiyedun wants to merge 6 commits into
Draft
Backport #23646, #23732, #23714: Fix cudf-java native load#23766paul-aiyedun wants to merge 6 commits into
paul-aiyedun wants to merge 6 commits into
Conversation
* Move `java-build` onto RAPIDS `ci-wheel` (Rocky 8) via `custom-job` and the shared `conda-cpp-build` matrix, replacing the previous host-side ci-conda docker wrappers. * Add `java-build` and `java-tests` jobs that run the existing suite against the uploaded classifier JAR. * Introduce `-Ppackaged-jar-tests` so Surefire uses the packaged JAR instead of `target/classes`, with `PackagedJarOriginCheck` to fail fast if classes load from the wrong place. * Factor classifier/image/env helpers (`java_classifier.sh`, `ci_wheel_image.sh`, `setup_java_env.sh`) and in-container entrypoints so local wrappers and CI share one packaging path. * Skip optional `nvcomp` in `NativeDepsLoader` when its native resource is absent, filtering that stage out of the load order by name. Static classifier JARs embed nvcomp in `libcudf.so` and ship no separate `libnvcomp.so`, which previously broke packaged-JAR startup with `UnsatisfiedLinkError`. Closes NVIDIA#23563 Authors: - https://github.com/paul-aiyedun Approvers: - Bradley Dice (https://github.com/bdice) - Tim Liu (https://github.com/NvTimLiu) - MithunR (https://github.com/mythrocks) URL: NVIDIA#23646 (cherry picked from commit 7c536bf)
* Move the `java-build` artifact download into `ci/test_packaged_java.sh` so PR and nightly runs call the same script. * Download `cudf_java_$(arch)_cu$(cuda_major)` via `rapids-download-from-github`. * Replace the inline `pr.yaml` `java-tests` job with `shared-workflows/custom-job.yaml`. * Add` java-build-matrix` and `java-tests` to `test.yaml`. Authors: - https://github.com/paul-aiyedun Approvers: - Bradley Dice (https://github.com/bdice) URL: NVIDIA#23732 (cherry picked from commit 30fa357)
* Add a tag-gated `java-publish` job that deploys the gathered maven repo as an RC through shared-workflows, with GPG signing and Sonatype staging for manual Central publish. * On release-tag builds, strip `-SNAPSHOT` and rewrite `java/pom.xml` so packaged artifacts use the release version. * Copy the `cuda12` classifier JAR as the unclassified primary so consumers of `ai.rapids:cudf` without a classifier still resolve. * Forward `GITHUB_REF` into the jar-build container so the release-tag check can see it and switch on release versioning. * Make `HOST_UID`/`HOST_GID` required and always chown outputs on EXIT. * Update `java/ci/README.md` for release vs SNAPSHOT versioning, sources/javadoc jars, and the unclassified primary layout. Authors: - https://github.com/paul-aiyedun Approvers: - Tim Liu (https://github.com/NvTimLiu) - Bradley Dice (https://github.com/bdice) URL: NVIDIA#23714 (cherry picked from commit e36fd8a)
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Contributor
Author
|
/ok to test 3685dc5 |
github-actions
Bot
added
libcudf
Contributor
Author
|
/ok to test a0be043 |
paul-aiyedun
added
bug
Fix the checks job’s pre-commit failure by pinning Cython to the version used in the 26.08 CI.
github-actions
Bot
added
libcudf
Contributor
Author
|
/ok to test 1ca0d20 |
Contributor
Author
|
/ok to test 3fb18bc |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Rebuild classifier JARs on RAPIDS
ci-wheel(Rocky 8) so the bundledJNI library links against a libstdc++ ABI that Ubuntu 22.04 and Rocky
Linux 8 can satisfy.
Skip optional
nvcompinNativeDepsLoaderwhen its native resourceis absent.
Run the existing Java suite against the packaged classifier JAR in PR
and nightly CI so tests cover the published artifact.
Add a tag-gated
java-publishjob that uploads the gathered Mavenrepo to Maven Central.
Checklist