libLLVM.a doesn't exist, so fix how enzyme is linking LLVM

[ZuseZ4]

There are a couple of bugs in both Enzyme and LLVM, so it took me a while to figure this one out.

1. linking against "LLVM" as a constant string only works if there's some type of libLLVM.something.
   We only have a libLLVM.so if we build LLVM as a dynamically linked library, which rust does e.g. for x86-64.
   There is no libLLVM.a equivalent otherwise, just a lot of individual archives.

2. In theory, the right thing to fix this would be

llvm_map_components_to_libnames(llvm_libs all)
target_link_libraries(Enzyme-${LLVM_VERSION_MAJOR} ${llvm_libs})

However, "all" isn't being handled correctly, see llvm/llvm-project#46347
It seems to be a common experience that some cmake macros are unreliable, so I was recommended
to just call the llvm-config binary.

    execute_process(COMMAND ${LLVM_TOOLS_BINARY_DIR}/llvm-config --libs all
                OUTPUT_VARIABLE llvm_libraries)

Trying to also add the --ldflags fails the subsequent parsing:

CMake Error:
  Running

   '/usr/bin/ninja' '-C' '/home/manuel/prog/rust-new/build/x86_64-unknown-linux-gnu/enzyme/build' '-t' 'recompact'

  failed with:

   ninja: error: build.ninja:606: expected '=', got identifier

  -lLLVMWindowsManifest -lLLVMXRay -lLLVMLibDriver -lLLVMDlltoolDriver -lL...

                        ^ near here

so we have to add the -L somedir in yet another command.

4. We therefore call

    set_target_properties(Enzyme-${LLVM_VERSION_MAJOR} PROPERTIES
        LINK_FLAGS "`${LLVM_TOOLS_BINARY_DIR}/llvm-config --ldflags`")

which correctly gives e.g. on my system -L/home/manuel/prog/rust-new/build/x86_64-unknown-linux-gnu/llvm/lib, where all the libLLVMSomething.a files are located.

So this still seems messy, but at least it should be more often correct than the current code. I'd be happy if someone else can fix it in upstream.

Alternative: Being more precise and only specifying llvm_map_components_to_libnames(llvm_libraries Passes) instead of trying to use all seems to work. But the recommendation seems to be the binary, so idk.

[ZuseZ4]

Also there is at least a 4th bug, but I think that one is on the rust side, where we at times pass the wrong llvm_dir (in cases like PGO builds, where we have multiple).

[ZuseZ4]

out.ll.txt

[ZuseZ4]

@wsmoses I don't understand how the julia build environment differs, especially the last commit was just a guess (since I don't know why zlib was previously commented out). However, it works. I'd probably try to again remove

        llvm_map_components_to_libnames(llvm_librariess Passes Support)
        target_link_libraries(Enzyme-${LLVM_VERSION_MAJOR} ${llvm_librariess})

once CI passed, given that I suspect that the ZLIB removal was the likely culprit for the build failure. Then, we'd be fully relying on llvm-config. Interestingly, rust CI also failed without the zlib change (even though it worked locally) https://github.com/rust-lang-ci/rust/actions/runs/14826389971/job/41620030778

Can you look into the precompilation failure? Also the Ubuntu runner got cancelled after an hour, not sure why. I didn't see any failures.

[wsmoses]

I'm fairly certain this was intentionally removed due to a bug in LLVM's packaging of the dylib

[ZuseZ4]

How do you work around it, since linking failed on the Julia side without this line.
Also, any other requests before landing this?

[ZuseZ4]

(I am also OK with having another switch, so julia can keep it's old code path.)
Rust can deal with upstream LLVM if the zlib packaging comes back to cause us issues, in case that its' still the case. I feel like Julia should probably still move over since I think it currently only works by coincidence, see my first point above, but I don't want to debug julia precompilation since I already have enough cmake and bootstrap challenges.

[wsmoses]

let's do these things one at a time to confirm they're fine.

Can you leave this as a non-functional change (aka keep old zlib behavior) first, but add the non dylib linking, and then a follow up for zlib change?

[ZuseZ4]

The problem is that I only added the zlib change, because Julia CI was failing (locally it worked fine without the patch). See https://github.com/EnzymeAD/Enzyme/actions/runs/14826186325/job/41619569312
For some reason it doesn't seem to pick up the /usr zlib library?