Skip to content

Update Clang in MRBind to 20. #4750

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build-test-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ jobs:
if: ${{inputs.mrbind}}
shell: msys2 {0}
run: |
./scripts/mrbind/msys2_install_clang_ver.sh $(cat scripts/mrbind/clang_version_msys2.txt)
./scripts/mrbind/msys2_install_clang_ver.sh $(scripts/mrbind/select_clang_version.sh)

- name: Install Ninja for CMake
if: ${{ matrix.build_system == 'CMake' }}
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,7 @@ jobs:
- 'scripts/install_apt_requirements.sh'
- 'scripts/install_dnf_requirements.sh'
- 'scripts/mrbind/install_deps_ubuntu.sh'
- 'scripts/mrbind/select_clang_version.sh'
- 'thirdparty/!(install.bat|vcpkg/**)'

- name: Filter Linux vcpkg paths
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pip-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,7 @@ jobs:
- name: Install Clang in MSYS2
shell: msys2 {0}
run: |
./scripts/mrbind/msys2_install_clang_ver.sh $(cat scripts/mrbind/clang_version_msys2.txt)
./scripts/mrbind/msys2_install_clang_ver.sh $(scripts/mrbind/select_clang_version.sh)

- name: Build MRBind
shell: cmd
Expand Down
2 changes: 1 addition & 1 deletion docker/ubuntu20Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ WORKDIR "/usr/local/lib/meshlib-thirdparty-lib/"

COPY scripts/install_apt_requirements.sh scripts/install_apt_requirements.sh
COPY scripts/mrbind/install_deps_ubuntu.sh scripts/mrbind/install_deps_ubuntu.sh
COPY scripts/mrbind/clang_version.txt scripts/mrbind/clang_version.txt
COPY scripts/mrbind/select_clang_version.sh scripts/mrbind/select_clang_version.sh
COPY scripts/install_thirdparty.sh scripts/install_thirdparty.sh
COPY scripts/patches scripts/patches
COPY requirements requirements
Expand Down
2 changes: 1 addition & 1 deletion docker/ubuntu22Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ WORKDIR "/usr/local/lib/meshlib-thirdparty-lib/"

COPY scripts/install_apt_requirements.sh scripts/install_apt_requirements.sh
COPY scripts/mrbind/install_deps_ubuntu.sh scripts/mrbind/install_deps_ubuntu.sh
COPY scripts/mrbind/clang_version.txt scripts/mrbind/clang_version.txt
COPY scripts/mrbind/select_clang_version.sh scripts/mrbind/select_clang_version.sh
COPY scripts/install_thirdparty.sh scripts/install_thirdparty.sh
COPY scripts/patches scripts/patches
COPY requirements requirements
Expand Down
2 changes: 1 addition & 1 deletion docker/ubuntu24Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ WORKDIR "/usr/local/lib/meshlib-thirdparty-lib/"

COPY scripts/install_apt_requirements.sh scripts/install_apt_requirements.sh
COPY scripts/mrbind/install_deps_ubuntu.sh scripts/mrbind/install_deps_ubuntu.sh
COPY scripts/mrbind/clang_version.txt scripts/mrbind/clang_version.txt
COPY scripts/mrbind/select_clang_version.sh scripts/mrbind/select_clang_version.sh
COPY scripts/install_thirdparty.sh scripts/install_thirdparty.sh
COPY scripts/patches scripts/patches
COPY requirements requirements
Expand Down
1 change: 1 addition & 0 deletions requirements/ubuntu.txt
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ libtinyxml2-dev
libturbojpeg0-dev
libzip-dev
libzstd-dev
lsb-release
occt-misc
pkg-config
python3-venv
12 changes: 6 additions & 6 deletions scripts/mrbind/README-generating.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Among other things, the scripts can do following:

We use [MSYS2 CLANG64](https://www.msys2.org/docs/environments/) environment. Consult `install_deps_windows_msys2.bat` for the list of packages we install in it.

We don't use the latest Clang version, instead we download and install the version specified in `clang_version_msys2.txt`.
We don't use the latest Clang version, instead we download and install the version specified in `select_clang_version.sh`.

* **Building MRBind:**

Expand All @@ -45,7 +45,7 @@ Among other things, the scripts can do following:

* **Installing dependencies:**

We want a certain version of Clang (see `clang_version.txt`), and since older versions of Ubuntu don't have it, we add Clang's repository: https://apt.llvm.org
We want a certain version of Clang (see `select_clang_version.sh`), and since older versions of Ubuntu don't have it, we add Clang's repository: https://apt.llvm.org

And obviously we install some packages, see `install_deps_ubuntu.sh` for the list.

Expand All @@ -55,7 +55,7 @@ Among other things, the scripts can do following:

We build MRBind at `MeshLib/mrbind`, but you can build it [elsewhere](#less-common-flags) manually.

You might want to pass `-DClang_DIR=/usr/lib/cmake/clang-VERSION` (where `VERSION` is the one mentioned in `clang_version.txt`) if you have several versions of libclang installed, because otherwise CMake might pick an arbitrary one (apparently it picks the first one returned by globbing `clang-*`, which might not be the latest one).
You might want to pass `-DClang_DIR=/usr/lib/cmake/clang-VERSION` (where `VERSION` is the one mentioned in `select_clang_version.sh`) if you have several versions of libclang installed, because otherwise CMake might pick an arbitrary one (apparently it picks the first one returned by globbing `clang-*`, which might not be the latest one).

Use `CC=clang-VERSION CXX=clang++-VERSION cmake ....` to build using Clang. Other compilers might work, but that's not guaranteed.

Expand All @@ -67,14 +67,14 @@ Among other things, the scripts can do following:

Homebrew must already be installed.

We install a certain version of Clang and libclang from it (see `clang_version.txt`), and also GNU Make and Gawk. MacOS has its own Make, but it's outdated. It seems to have Gawk, but we install our own just in case.
We install a certain version of Clang and libclang from it (see `select_clang_version.sh`), and also GNU Make and Gawk. MacOS has its own Make, but it's outdated. It seems to have Gawk, but we install our own just in case.

What we install from Brew is the regular Clang, not Apple Clang (Apple's fork for Clang), because that is based on an outdated branch of Clang.

You must run following to add the installed things to your PATH. On Arm Macs:
```sh
export PATH="/opt/homebrew/opt/make/libexec/gnubin:$PATH"
export PATH="/opt/homebrew/opt/llvm/bin@VERSION:$PATH" # See the correct VERSION in `clang_version.txt`.
export PATH="/opt/homebrew/opt/llvm/bin@VERSION:$PATH" # See the correct VERSION in `select_clang_version.sh`.
```
And on x86 Macs the installation directory seems to be `/usr/local/...` instead of `/opt/homebrew/...`.

Expand Down Expand Up @@ -123,7 +123,7 @@ Then generate the bindings:

### Selecting the compiler:

For simplicity, we compile the bindings with the same Clang that we use for parsing the code. (Consult `clang_version.txt` for the current version.) But you can override this using `CXX_FOR_BINDINGS`.
For simplicity, we compile the bindings with the same Clang that we use for parsing the code. (Consult `select_clang_version.sh` for the current version.) But you can override this using `CXX_FOR_BINDINGS`.

**ABI compatibility:** Since MeshLib is compiled using a different compiler, we must ensure the two use the same ABI. `CXX_FOR_ABI` should be set to the compiler the ABI of which we're trying to match. (Defaults to `CXX` environment variable, or `g++` if not set.) At the moment, if `CXX_FOR_ABI` is GCC 13 or older or Clang 17 or older (note that Apple Clang uses a [different version numbering scheme](https://en.wikipedia.org/wiki/Xcode#Xcode_15.0_-_(since_visionOS_support)_2)), we pass `-fclang-abi-compat=17` to our Clang 18 or newer. This flag *disables* mangling `requires` constraints into function names. If we guess incorrectly, you'll get undefined references to functions with `requires` constraints on them.

Expand Down
2 changes: 1 addition & 1 deletion scripts/mrbind/clang_version.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
18
20
2 changes: 1 addition & 1 deletion scripts/mrbind/clang_version_msys2.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
18.1.8-2
20.1.7-1
2 changes: 0 additions & 2 deletions scripts/mrbind/common_compiler_parser_flags.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
-std=c++23
-Wno-nonportable-include-path
-Wno-enum-constexpr-conversion
-Wno-deprecated-enum-enum-conversion
-frelaxed-template-template-args
10 changes: 8 additions & 2 deletions scripts/mrbind/generate.mk
Original file line number Diff line number Diff line change
Expand Up @@ -156,10 +156,10 @@ MRBIND_EXE := $(MRBIND_SOURCE)/build/mrbind
ifneq ($(IS_WINDOWS),)
CXX_FOR_BINDINGS := clang++
else ifneq ($(IS_MACOS),)
CXX_FOR_BINDINGS := $(HOMEBREW_DIR)/opt/llvm@$(strip $(file <$(makefile_dir)clang_version.txt))/bin/clang++
CXX_FOR_BINDINGS := $(HOMEBREW_DIR)/opt/llvm@$(call safe_shell,$(makefile_dir)select_clang_version.sh)/bin/clang++
else
# Only on Ubuntu we don't want the default Clang version, as it can be outdated. Use the suffixed one.
CXX_FOR_BINDINGS := clang++-$(strip $(file <$(makefile_dir)clang_version.txt))
CXX_FOR_BINDINGS := clang++-$(call safe_shell,$(makefile_dir)select_clang_version.sh)
endif

# Which C++ compiler we should try to match for ABI.
Expand Down Expand Up @@ -428,7 +428,13 @@ INPUT_GLOBS := *.h
# `referenced by source/TempOutput/PythonBindings/x64/Release/binding.0.o:(public: __cdecl std::_Literal_zero::_Literal_zero<int>(int))`.
MRBIND_FLAGS := $(call load_file,$(makefile_dir)mrbind_flags.txt)
MRBIND_FLAGS_FOR_EXTRA_INPUTS := $(call load_file,$(makefile_dir)mrbind_flags_for_helpers.txt)

COMPILER_FLAGS := $(ABI_COMPAT_FLAG) $(EXTRA_CFLAGS) $(call load_file,$(makefile_dir)common_compiler_parser_flags.txt) -I. -I$(DEPS_INCLUDE_DIR) -I$(makefile_dir)../../source
# Add some flags for old Clangs.
ifneq ($(filter 18 19,$(call safe_shell,$(CXX_FOR_BINDINGS) -dumpversion | cut -d. -f1)),)
COMPILER_FLAGS += -Wno-enum-constexpr-conversion -frelaxed-template-template-args
endif

COMPILER_FLAGS_LIBCLANG := $(call load_file,$(makefile_dir)parser_only_flags.txt)
# Need whitespace before `$(MRBIND_SOURCE)` to handle `~` correctly.
COMPILER := $(CXX_FOR_BINDINGS) $(subst $(lf), ,$(call load_file,$(makefile_dir)compiler_only_flags.txt)) -I $(MRBIND_SOURCE)/include -I$(makefile_dir)
Expand Down
4 changes: 1 addition & 3 deletions scripts/mrbind/install_deps_macos.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,8 @@
# We assume `brew` is already installed. Automatic its installation is too much,
# especially because of the conflicts that happen if several users install it.

# Read the Clang version from `clang_version.txt`. `xargs` trims the whitespace.
# Some versions of MacOS seem to lack `realpath`, so not using it here.
SCRIPT_DIR="$(dirname "$BASH_SOURCE")"
CLANG_VER="$(cat $SCRIPT_DIR/clang_version.txt | xargs)"
CLANG_VER="$("$SCRIPT_DIR/select_clang_version.sh")"
[[ $CLANG_VER ]] || (echo "Not sure what version of Clang to use." && false)

brew update
Expand Down
3 changes: 1 addition & 2 deletions scripts/mrbind/install_deps_ubuntu.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,8 @@ apt update
# Install `xargs` because we need it below.
apt install -y findutils

# Read the Clang version from `clang_version.txt`. `xargs` trims the whitespace.
SCRIPT_DIR="$(realpath "$(dirname "$BASH_SOURCE")")"
CLANG_VER="$(cat $SCRIPT_DIR/clang_version.txt | xargs)"
CLANG_VER="$("$SCRIPT_DIR/select_clang_version.sh")"
[[ $CLANG_VER ]] || (echo "Not sure what version of Clang to use." && false)

# Add LLVM repositories if the required package is not accessible right now.
Expand Down
3 changes: 1 addition & 2 deletions scripts/mrbind/install_deps_windows_msys2.bat
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ rem some other MSYS2 copy.
setlocal

if "%MSYS2_DIR%" == "" set MSYS2_DIR=C:\msys64_meshlib_mrbind
if "%CLANG_VER%" == "" set /p CLANG_VER=<%~dp0\clang_version_msys2.txt

rem ------ Ensure MSYS2 is installed

Expand Down Expand Up @@ -50,6 +49,6 @@ rem ------ Install needed packages
call %MSYS2_DIR%\msys2_shell.cmd -no-start -defterm -clang64 -c "pacman -S --noconfirm --needed gawk make procps-ng $MINGW_PACKAGE_PREFIX-cmake"

rem ------ Install a specific version of Clang
call %MSYS2_DIR%\msys2_shell.cmd -no-start -defterm -clang64 -c "'%~dp0'/msys2_install_clang_ver.sh %CLANG_VER%"
call %MSYS2_DIR%\msys2_shell.cmd -no-start -defterm -clang64 -c "'%~dp0'/msys2_install_clang_ver.sh $('%~dp0'/select_clang_version.sh)"

endlocal
4 changes: 1 addition & 3 deletions scripts/mrbind/install_mrbind_macos.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,7 @@ SCRIPT_DIR="$(dirname "$BASH_SOURCE")"

[[ ${MRBIND_DIR:=} ]] || MRBIND_DIR="$SCRIPT_DIR/../../thirdparty/mrbind"

# Read the Clang version from `clang_version.txt`. `xargs` trims the whitespace.
# Some versions of MacOS seem to lack `realpath`, so not using it here.
CLANG_VER="$(cat "$SCRIPT_DIR/clang_version.txt" | xargs)"
CLANG_VER="$("$SCRIPT_DIR/select_clang_version.sh")"
[[ ${CLANG_VER:=} ]] || (echo "Not sure what version of Clang to use." && false)

cd "$MRBIND_DIR"
Expand Down
3 changes: 1 addition & 2 deletions scripts/mrbind/install_mrbind_ubuntu.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ SCRIPT_DIR="$(realpath "$(dirname "$BASH_SOURCE")")"

[[ -v MRBIND_DIR ]] || MRBIND_DIR="$(realpath "$SCRIPT_DIR/../../thirdparty/mrbind")"

# Read the Clang version from `clang_version.txt`. `xargs` trims the whitespace.
CLANG_VER="$(cat "$SCRIPT_DIR/clang_version.txt" | xargs)"
CLANG_VER="$("$SCRIPT_DIR/select_clang_version.sh")"
[[ $CLANG_VER ]] || (echo "Not sure what version of Clang to use." && false)

cd "$MRBIND_DIR"
Expand Down
63 changes: 63 additions & 0 deletions scripts/mrbind/select_clang_version.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
#!/bin/bash

# Returns the version of Clang that we want to use on the current platform.
# If you also pass `--flags`, we return the extra flags it needs.

set -euo pipefail

if [[ $# > 0 ]]; then
echo 'Expected no arguments.' >&2
exit
fi


CLANG_18=18
CLANG_20=20
CLANG_MSYS2=20.1.7-1


# Windows, Clang installed via MSYS2.
if [[ $(uname -o) == Msys ]]; then
echo $CLANG_MSYS2
exit
fi

UNAME_S=$(uname -s)

# MacOS.
if [[ $UNAME_S == Darwin ]]; then
# Clang 20 can't find the C++ standard library, so staying 18 for now.
# I didn't dig too deep here, and we have to maintain 18 anyway for Ubuntu 20.04 and 22.04 (see below for why that is).
echo 18
exit
fi

# Linux.
if [[ $UNAME_S == Linux ]]; then
# Here we use `type` to not rely on `which` existing, since it's getting removed from some distros.
if ! type lsb_release >/dev/null 2>/dev/null; then
echo "`lsb_release` is not installed!" >&2
fi

# Is `Ubuntu <= 22.04`?
if (lsb_release -rs; echo "22.04") | sort -CV; then
# Here we need the outdated Clang because the old Boost doesn't compile on the new Clang, because of this change: https://github.com/llvm/llvm-project/issues/59036
# This is what is actually locking us to Clang 18 at the moment. Other platforms are using it for less scare reasons.
echo 18
exit
fi

# Is any other ubuntu?
if [[ $(lsb_release -is) == Ubuntu ]]; then
# This is because teh stock spdlog and libfmt fail with `call to consteval function ... is not a constant expression` somewhere in the formatting logic. Hmm.
echo 18
exit
fi

# Any other linux.
echo 20
exit
fi

echo "Unknown OS" >&2
exit 1
Loading