build-scripts/build_llvm.py: bump to llvm 18 #4259

yamt · 2025-05-08T04:38:55Z

why not 20?
because, as of writing this, 19 is the latest released version for
the xtensa fork of llvm: https://github.com/espressif/llvm-project

why not 19?
because of a bug in the xtensa fork of llvm:
espressif/llvm-project#112

while we can use different versions for different targets,
it's nicer to use the same version everywhere when possible.

cf. bytecodealliance#4210 why not 20? because, as of writing this, 19 is the latest released version for the xtensa fork of llvm: https://github.com/espressif/llvm-project why not 19? because of a bug in the xtensa fork of llvm: espressif/llvm-project#112 while we can use different versions for different targets, it's nicer to use the same version everywhere when possible.

with the recent version of LLVM, wamrc --size-level=1 often generates R_X86_64_32S relocations which fail on load with the infamous error: "relocation truncated to fit R_X86_64_32S failed" it seems that these relocations are often for jump tables. this commit workarounds it with --size-level=0. an alternative is to disable jump tables. (although it seems that jump tables are not the only source of these relocations.) cf. bytecodealliance#3035 it might be better to do this in wamrc itself. however, currently target info is not available there in case of native compilation. related: bytecodealliance#3356

cf. bytecodealliance#3035

lum1n0us

LGTM

TianlongLiang

LGTM

lum1n0us · 2025-05-12T03:59:53Z

@yamt

I made a test run on my forked repository and encountered two failures related to LLVM compilation:

https://github.com/lum1n0us/wasm-micro-runtime/actions/runs/14962486268/job/42026672520. Like: clang: error: no such file or directory: '/Users/runner/work/wasm-micro-runtime/wasm-micro-runtime/core/deps/llvm-project/build/tools/lldb/tools/debugserver/source/mach_excUser.c'
https://github.com/lum1n0us/wasm-micro-runtime/actions/runs/14962486268/job/42028726543. Like: D:\a\wasm-micro-runtime\wasm-micro-runtime\core\deps\llvm\build\include\llvm\ADT\STLForwardCompat.h(46,35): error C2039: 'optional': is not a member of 'std'

yamt · 2025-05-12T06:06:50Z

@yamt

I made a test run on my forked repository and encountered two failures related to LLVM compilation:

* https://github.com/lum1n0us/wasm-micro-runtime/actions/runs/14962486268/job/42026672520. Like: _clang: error: no such file or directory: '/Users/runner/work/wasm-micro-runtime/wasm-micro-runtime/core/deps/llvm-project/build/tools/lldb/tools/debugserver/source/mach_excUser.c'_

i'm not sure how it can happen off hand.
those files are expected to be generated at:
https://github.com/lum1n0us/wasm-micro-runtime/actions/runs/14962486268/job/42026672520#step:14:26302

* https://github.com/lum1n0us/wasm-micro-runtime/actions/runs/14962486268/job/42028726543. Like: _D:\a\wasm-micro-runtime\wasm-micro-runtime\core\deps\llvm\build\include\llvm\ADT\STLForwardCompat.h(46,35): error C2039: 'optional': is not a member of 'std'_

maybe #4269 ?
just a wild guess, i haven't used wamr on windows.

lum1n0us · 2025-05-12T09:17:43Z

https://github.com/lum1n0us/wasm-micro-runtime/actions/runs/14967033117/job/42039634773

Maybe we missed a part that needs to adapt build-scripts/lldb_wasm.patch to LLVM 18?

In my personal repository, I have updated LLVM in .github/workflows/build_wamr_lldb.yml to 18.1.8, and the step "apply WAMR patch" failed.

      - name: download llvm
        if: steps.lldb_build_cache.outputs.cache-hit != 'true'
        run: |
          wget -O llvm-project.tar.xz --progress=dot:giga https://github.com/llvm/llvm-project/releases/download/llvmorg-18.1.8/llvm-project-18.1.8.src.tar.xz
          tar -xzf llvm-project.tar.xz
          mv llvm-project-18.1.8.src llvm-project
        working-directory: core/deps/

yamt · 2025-05-12T22:41:53Z

https://github.com/lum1n0us/wasm-micro-runtime/actions/runs/14967033117/job/42039634773

Maybe we missed a part that needs to adapt build-scripts/lldb_wasm.patch to LLVM 18?

In my personal repository, I have updated LLVM in .github/workflows/build_wamr_lldb.yml to 18.1.8, and the step "apply WAMR patch" failed.
      - name: download llvm
        if: steps.lldb_build_cache.outputs.cache-hit != 'true'
        run: |
          wget -O llvm-project.tar.xz --progress=dot:giga https://github.com/llvm/llvm-project/releases/download/llvmorg-18.1.8/llvm-project-18.1.8.src.tar.xz
          tar -xzf llvm-project.tar.xz
          mv llvm-project-18.1.8.src llvm-project
        working-directory: core/deps/

oops, i have completely forgotten that patch. :-)

@xujuntwt95329 do you remember what's the status of llvm/llvm-project#77949
and https://github.com/xujuntwt95329/wasm-micro-runtime/tree/wasm_debug_2024 ?

lum1n0us · 2025-05-13T05:13:23Z

i'm not sure how it can happen off hand.those files are expected to be generated at:

Those files are not able to be generated.

set(generated_mach_interfaces
  ${CMAKE_CURRENT_BINARY_DIR}/mach_exc.h
  ${CMAKE_CURRENT_BINARY_DIR}/mach_excServer.c
  ${CMAKE_CURRENT_BINARY_DIR}/mach_excUser.c
  )

#...

add_custom_command(OUTPUT ${generated_mach_interfaces}
  VERBATIM COMMAND mig ${MIG_ARCH_FLAGS_SEPARTED} -isysroot ${CMAKE_OSX_SYSROOT} ${CMAKE_CURRENT_SOURCE_DIR}/MacOSX/dbgnub-mig.defs
  DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/MacOSX/dbgnub-mig.defs
  )

Based on the running command. from https://github.com/lum1n0us/wasm-micro-runtime/actions/runs/14988324213/job/42106372313#step:12:10024. ${CMAKE_OS_SYSROOT} isn't set properly. It will let mach/mach_exc.defs can't be found which is imported by dbgnub-mig.defs.

[368/407] cd /Users/runner/work/wasm-micro-runtime/wasm-micro-runtime/core/deps/llvm-project/build/tools/lldb/tools/debugserver/source && mig -isysroot /Users/runner/work/wasm-micro-runtime/wasm-micro-runtime/core/deps/llvm-project/lldb/tools/debugserver/source/MacOSX/dbgnub-mig.defs

lum1n0us · 2025-05-13T05:20:03Z

It is a known issue with CMake 4.0. llvm/llvm-project#138020

#4274

* build-scripts/build_llvm.py: bump to llvm 18 cf. bytecodealliance#4210 why not 20? because, as of writing this, 19 is the latest released version for the xtensa fork of llvm: https://github.com/espressif/llvm-project why not 19? because of a bug in the xtensa fork of llvm: espressif/llvm-project#112 while we can use different versions for different targets, it's nicer to use the same version everywhere when possible. * spec-test-script/runtest.py: --size-level=0 for x86-64 with the recent version of LLVM, wamrc --size-level=1 often generates R_X86_64_32S relocations which fail on load with the infamous error: "relocation truncated to fit R_X86_64_32S failed" it seems that these relocations are often for jump tables. this commit workarounds it with --size-level=0. an alternative is to disable jump tables. (although it seems that jump tables are not the only source of these relocations.) cf. bytecodealliance#3035 it might be better to do this in wamrc itself. however, currently target info is not available there in case of native compilation. related: bytecodealliance#3356 * wamr-compiler: size_level=0 for sgx mode cf. bytecodealliance#3035

lum1n0us · 2025-05-14T02:20:29Z

@yamt @xujuntwt95329 Is there any chance that we can skip lldb in the v2.3.0 release? I've learned that upgrading lldb_wasm.patch to LLVM 18 will not be a quick and smooth process. Maybe we shouldn't let v2.3.0 wait for it?

yamt · 2025-05-14T04:31:36Z

@yamt @xujuntwt95329 Is there any chance that we can skip lldb in the v2.3.0 release? I've learned that upgrading lldb_wasm.patch to LLVM 18 will not be a quick and smooth process. Maybe we shouldn't let v2.3.0 wait for it?

maybe just leave it as it is? (ie llvm 13)
iirc, it's what we did when upgrading to llvm 15.

yamt force-pushed the llvm-18-2 branch from 4908c4e to 8b7a83a Compare May 8, 2025 05:28

yamt added 2 commits May 8, 2025 15:22

wamr-compiler: size_level=0 for sgx mode

a6e8138

cf. bytecodealliance#3035

yamt changed the title ~~test llvm 18 for xtensa~~ build-scripts/build_llvm.py: bump to llvm 18 May 8, 2025

yamt marked this pull request as ready for review May 8, 2025 08:01

yamt requested review from loganek, lum1n0us, no1wudi, TianlongLiang, wenyongh and xujuntwt95329 as code owners May 8, 2025 08:01

yamt mentioned this pull request May 8, 2025

build-scripts/build_llvm.py: bump to llvm 19 #4213

Draft

lum1n0us approved these changes May 9, 2025

View reviewed changes

TianlongLiang approved these changes May 9, 2025

View reviewed changes

lum1n0us merged commit 908838a into bytecodealliance:main May 9, 2025
402 checks passed

lum1n0us mentioned this pull request May 12, 2025

Update version to 2.3.0 #4171

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

build-scripts/build_llvm.py: bump to llvm 18 #4259

build-scripts/build_llvm.py: bump to llvm 18 #4259

Uh oh!

yamt commented May 8, 2025 •

edited

Loading

Uh oh!

lum1n0us left a comment

Uh oh!

TianlongLiang left a comment

Uh oh!

Uh oh!

lum1n0us commented May 12, 2025

Uh oh!

yamt commented May 12, 2025

Uh oh!

lum1n0us commented May 12, 2025

Uh oh!

yamt commented May 12, 2025

Uh oh!

lum1n0us commented May 13, 2025

Uh oh!

lum1n0us commented May 13, 2025 •

edited

Loading

Uh oh!

lum1n0us commented May 14, 2025

Uh oh!

yamt commented May 14, 2025

Uh oh!

Uh oh!

build-scripts/build_llvm.py: bump to llvm 18 #4259

build-scripts/build_llvm.py: bump to llvm 18 #4259

Uh oh!

Conversation

yamt commented May 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lum1n0us left a comment

Choose a reason for hiding this comment

Uh oh!

TianlongLiang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lum1n0us commented May 12, 2025

Uh oh!

yamt commented May 12, 2025

Uh oh!

lum1n0us commented May 12, 2025

Uh oh!

yamt commented May 12, 2025

Uh oh!

lum1n0us commented May 13, 2025

Uh oh!

lum1n0us commented May 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lum1n0us commented May 14, 2025

Uh oh!

yamt commented May 14, 2025

Uh oh!

Uh oh!

yamt commented May 8, 2025 •

edited

Loading

lum1n0us commented May 13, 2025 •

edited

Loading