Skip to content

Conversation

@3manifold
Copy link

@3manifold 3manifold commented Jul 17, 2025

Description

manylinux2014 and rhel8 incompatibility

The manylinux2014 images (see .github/workflows/ci.yml) are based on CentOS 7, which is a RedHat Enterprise Linux (RHEL) derivative. In specific:

manylinux2014 → Based on CentOS 7
CentOS 7 → Based on RHEL 7

Nevertheless, python/tools/prepare_build_environment_linux.sh assumes RHEL 8 repository (see yum-config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo).

This causes build-python-wheels (ubuntu-22.04, auto64) CI job to fail: the issue is that the libnccl package requires a newer version of libstdc++ that supports CXXABI_1.3.9, but the system (which appears to be CentOS 7 based on the gcc version 4.8.5-44.el7) has an older version of libstdc++.

The issue is possibly caused by the changes made in #1803 .

Error Log

...
  --> Processing Dependency: libstdc++.so.6(CXXABI_1.3.9)(64bit) for package: libnccl-2.27.6-1+cuda12.9.x86_64
  --> Running transaction check
  ---> Package cpp.x86_64 0:4.8.5-44.el7 will be installed
  ---> Package libnccl.x86_64 0:2.27.6-1+cuda12.9 will be installed
  --> Processing Dependency: libstdc++.so.6(CXXABI_1.3.9)(64bit) for package: libnccl-2.27.6-1+cuda12.9.x86_64
  --> Finished Dependency Resolution
  Error: Package: libnccl-2.27.6-1+cuda12.9.x86_64 (cuda-rhel8-x86_64)
             Requires: libstdc++.so.6(CXXABI_1.3.9)(64bit)
   You could try using --skip-broken to work around the problem
...

Unsupported Microsoft Server 2022 MSVC version

Had to install the preexisting (ms server 2019) compiler version. CUDA does not support Microsoft Server 2022 MSVC version in practice even if the error message states ..2022 (inclusive)..

Error Log

...
fatal error C1189: #error: -- unsupported Microsoft Visual Studio version! Only the versions between 2017 and 2022 (inclusive) are supported!
...

intel-oneapi download error

Download command curl --netrc-optional -L -nv -o webimage.exe https://registrationcenter-download.intel.com/akdlm/irc_nas/19078/w_BaseKit_p_2023.0.0.25940_offline.exe continuously fails (there are many reports on the internet on that wrt intel-oneapi). In general any URL having the following pattern (https://registrationcenter-download.intel.com/akdlm/irc_nas) fails to download after some point. Fixed by replacing with a modern api version.

resolves #1906

@3manifold 3manifold marked this pull request as ready for review July 17, 2025 21:14
@3manifold 3manifold marked this pull request as draft July 18, 2025 07:26
@3manifold 3manifold force-pushed the fix-ci-rhel8 branch 3 times, most recently from 127d711 to e367bfe Compare July 18, 2025 12:40
@3manifold 3manifold marked this pull request as ready for review July 18, 2025 12:54
@3manifold 3manifold force-pushed the fix-ci-rhel8 branch 2 times, most recently from 940857a to 8065dba Compare July 18, 2025 20:52
@3manifold 3manifold marked this pull request as draft July 18, 2025 21:50
@3manifold 3manifold force-pushed the fix-ci-rhel8 branch 2 times, most recently from 44606aa to 7897b9f Compare July 18, 2025 22:36
@3manifold 3manifold marked this pull request as ready for review July 18, 2025 22:42
@3manifold 3manifold force-pushed the fix-ci-rhel8 branch 2 times, most recently from c09f9da to 7762f1a Compare July 19, 2025 20:48
@3manifold 3manifold changed the title Fix incompatibility among manylinux2014 and rhel8 Fix CI failure due to manylinux2014 and rhel8 incompatibility, deprecated MS server 2019 & intel-oneapi download errors Jul 20, 2025
cp "C:/Program Files (x86)/Intel/oneAPI/compiler/latest/windows/redist/intel64_win/compiler/libiomp5md.dll" python/ctranslate2/

# Find the libiomp5md.dll file
LIBIOMP5_PATH=$(find "C:/Program Files (x86)/Intel/oneAPI" -name "libiomp5md.dll" -type f 2>/dev/null | head -1)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depending on the oneAPI version, .dll location may vary. This way one does not need to provide a hard-coded path.

@3manifold
Copy link
Author

yum install -y intel-oneapi-mkl-devel-$ONEAPI_VERSION could be also changed to intel-oneapi-mkl-devel-2025.1 in python/tools/prepare_build_environment_linux.sh to match python/tools/prepare_build_environment_windows.sh if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] CI fails due to manylinux2014 and rhel8 incompatibility, deprecated MS server 2019 & intel-oneapi download errors

2 participants