share object definition error when using wheel on python 3.10 and cuda 121 ( with 126 download )

[guitoud]

Prerequisites

python 3.10
cuda 121

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

Got an error : OSError: libcudart.so.12: cannot open shared object file: No such file or directory

I lanch : >python -c "from llama_cpp.llama_chat_format import Qwen3VLChatHandler, Qwen25VLChatHandler; print('handlers OK')"

Current Behavior

got this error
Traceback (most recent call last):
File "/app/server/projects/pulid/venv/lib/python3.10/site-packages/llama_cpp/_ctypes_extensions.py", line 82, in load_shared_library
return ctypes.CDLL(str(lib_path), **cdll_args) # type: ignore
File "/usr/local/lib/python3.10/ctypes/init.py", line 374, in init
self._handle = _dlopen(self._name, mode)
OSError: libcudart.so.12: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/app/server/projects//venv/lib/python3.10/site-packages/llama_cpp/init.py", line 1, in
from .llama_cpp import *
File "/app/server/projects//venv/lib/python3.10/site-packages/llama_cpp/llama_cpp.py", line 8, in
from ._ggml import (
File "/app/server/projects//venv/lib/python3.10/site-packages/llama_cpp/_ggml.py", line 21, in
libggml = ctypes_ext.load_shared_library("ggml", libggml_base_path)
File "/app/server/projects//venv/lib/python3.10/site-packages/llama_cpp/_ctypes_extensions.py", line 84, in load_shared_library
raise RuntimeError(f"Failed to load shared library '{lib_path}': {e}")
RuntimeError: Failed to load shared library '/app/server/projects/pulid/venv/lib/python3.10/site-packages/llama_cpp/lib/libggml.so': libcudart.so.12: cannot open shared object file: No such file or directory

Environment and Context

When I all this LD_LIBRARY_PATH =
$> LD_LIBRARY_PATH="/app/server/projects//venv/lib/python3.10/site-packages/nvidia/cuda_runtime/lib:/app/server/projects//venv/lib/python3.10/site-packages/nvidia/cublas/lib"

it is correctly launched

Please provide detailed information about your computer setup. This is important in case the issue is not reproducible except for under certain specific conditions.

• Physical (or virtual) hardware you are using, e.g. for Linux:

$ lscpu
lscpu
Architecture: x86_64
CPU(s): 12
Model name: AMD Ryzen 5 3600 6-Core Processor

• Operating System, e.g. for Linux:

Linux <conainer_id> 6.14.0-37-generic #37~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Nov 20 10:25:38 UTC 2 x86_64 GNU/Linux

• SDK version, e.g. for Linux:
  ubuntu 21.04

$ python3 --version
Python 3.10.13
$ make --version
GNU Make 4.3
$ g++ --version
g++ (Debian 12.2.0-14) 12.2.0

[JamePeng]

It's quite obvious that your CUDA environment is version 12.1, while the version you installed is 12.6. This mismatch in library versions is the direct cause of the missing libraries.The lowest version currently compiled is cu124.
I haven't compiled the CUDA 12.1 version of wheel on github action. You need to compile it locally.
You can see the instructions here: https://github.com/JamePeng/llama-cpp-python?tab=readme-ov-file#supported-backends

Linux

CMAKE_ARGS="-DGGML_CUDA=on" pip install "llama-cpp-python @ git+https://github.com/JamePeng/llama-cpp-python.git"