ERROR: llama_cpp_python_cuda-0.2.6+cu117-cp310-cp310-manylinux_2_31_x86_64.whl is not a supported wheel on this platform.

[jFkd1]

See the relevant issue with logs here: oobabooga/textgen#4005

Error about the wheel:

Ignoring llama-cpp-python: markers 'platform_system == "Windows"' don't match your environment
Ignoring llama-cpp-python-cuda: markers 'platform_system == "Windows"' don't match your environment
ERROR: llama_cpp_python_cuda-0.2.6+cu117-cp310-cp310-manylinux_2_31_x86_64.whl is not a supported wheel on this platform.

System: Ubuntu 22.04
CUDA: 11.7
Python: 3.10

[jllllll]

In the past, this was caused by trying to use the wrong Python version. You might want to make absolutely sure that you aren't unintentionally calling the wrong Python. For instance, pip install requirements.txt will use the system installed Python instead of Conda's due to Conda's Python package not providing a separate pip executable. The proper way to call the Conda Python installation is to use python -m pip install requirements.txt.

The alternative, if that isn't the issue, is much harder to solve. llama-cpp-python recently changed it's build backend to scikit-build-core, which uses the manylinux standard to define package compatibility. In the case of this wheel, it uses manylinux_2_31. This tells pip that the wheel is compatible with any version of Linux that uses glibc 2.31+.

In theory, this is better, but if that isn't working correctly then there may not be anything I can do about it. I'm building the wheels using the oldest version of Linux offered by GitHub Actions for better system compatibility (Ubuntu 20.04).

[creed2415]

ALSO OCCURS in CUDA:11.8, even I have tryed with CUDA:12.2
ERROR: llama_cpp_python_cuda-0.2.10+cu118-cp310-cp310-manylinux_2_31_x86_64.whl is not a supported wheel on this platform.
System: CENTOS7.9
CUDA: 11.7
Python: 3.10

[jllllll]

CentOS uses an older version of glibc, making it incompatible with the current builds.
I may investigate ways to support glibc 2.12, but I currently don't have the time.

CentOS 7.9 is a fairly old OS, released in 2009. I'm not even sure if it is worth supporting.
I'll look into it when I have the time.

[creed2415]

Thx，I figure it out with rockylinux9.2