Depend on NVML only, not the full CUDA toolkit - #77
Open
Mob97 wants to merge 2 commits into
Open
Conversation
gpu.c uses NVML (nvmlInit, nvmlDeviceGetMemoryInfo) -- the management library
that ships with the NVIDIA driver and backs nvidia-smi -- and never touches the
CUDA compute runtime. The build nevertheless required find_package(CUDA
REQUIRED) and linked -lcudart -lcublas, so a host with a perfectly good driver
could not build the GPU version without installing the entire toolkit. A grep
for cuda_runtime/cublas/cudaMalloc/cudaMemcpy across the tree finds no user of
either library outside that link line.
Add cmake/FindNVML.cmake, which looks for nvml.h and the nvidia-ml library in
driver locations directly and treats CUDA_HOME/CUDA_PATH/CUDA_ROOT as optional
search hints rather than requirements. Two cases needed care:
- Driver-only hosts often ship libnvidia-ml.so.1 without the unversioned
symlink find_library() looks for, so fall back to find_file() on the
versioned soname and link it by absolute path.
- WSL2 exposes the Windows driver under /usr/lib/wsl/lib, which is not a
standard library directory, so search it explicitly.
The Makefile now links -lnvidia-ml alone, with CUDA_HOME used only as a hint if
set. Verified on a driver-only host with no CUDA toolkit: the GPU build links
and `ldd ts` resolves libnvidia-ml.so.1 to the driver.
Document where nvml.h actually comes from. It is not part of any driver;
on Debian/Ubuntu it is libnvidia-ml-dev, whose only payload is
/usr/include/nvml.h. Also note the /usr/include/hwloc/nvml.h decoy, which is an
unrelated file that a naive search will find.
The TASK_SPOOLER_COMPILE_CUDA option name is kept as-is for backward
compatibility with existing build scripts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Neither README nor INSTALL.md said what to install. The GPU build in particular gave no hint that it needs nvml.h, which no NVIDIA driver ships -- a working nvidia-smi tells you nothing about build-time files. Split the list by build-time and run-time, since they differ sharply. The CPU binary links nothing but libc (verified with ldd), and the GPU binary adds only libnvidia-ml.so.1 from the driver. The remaining run-time entries are optional and tied to specific flags: gzip for -z (execute.c:128) and an MTA at /usr/sbin/sendmail for -m (mail.c:37). Note that the RHEL/Fedora NVML header package lives in NVIDIA's own CUDA repository rather than the base repositories, which is easy to trip over. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
gpu.c uses NVML (nvmlInit, nvmlDeviceGetMemoryInfo) -- the management library that ships with the NVIDIA driver and backs nvidia-smi -- and never touches the CUDA compute runtime. The build nevertheless required find_package(CUDA REQUIRED) and linked -lcudart -lcublas, so a host with a perfectly good driver could not build the GPU version without installing the entire toolkit. A grep for cuda_runtime/cublas/cudaMalloc/cudaMemcpy across the tree finds no user of either library outside that link line.
Add cmake/FindNVML.cmake, which looks for nvml.h and the nvidia-ml library in driver locations directly and treats CUDA_HOME/CUDA_PATH/CUDA_ROOT as optional search hints rather than requirements. Two cases needed care:
The Makefile now links -lnvidia-ml alone, with CUDA_HOME used only as a hint if set. Verified on a driver-only host with no CUDA toolkit: the GPU build links and
ldd tsresolves libnvidia-ml.so.1 to the driver.Document where nvml.h actually comes from. It is not part of any driver; on Debian/Ubuntu it is libnvidia-ml-dev, whose only payload is /usr/include/nvml.h. Also note the /usr/include/hwloc/nvml.h decoy, which is an unrelated file that a naive search will find.
The TASK_SPOOLER_COMPILE_CUDA option name is kept as-is for backward compatibility with existing build scripts.