Build IREE for iOS

[wangkuiyi]

I wish I could help make IREE work on iOS. So, I learned how to build IREE for the host and Android from the tutorial. @antiagainst told me that the IREE Vulcan backend could be a good example for Metal. Now, I think the first thing I should do is build IREE for iOS before adding new features like the Metal backend. I hope that IREE's CPU backend will help me with the first demo.

So, I tried to find a CMake command line for iOS comparable to the Android one. I took options for CMake from the following places.

1. https://github.com/light-tech/LLVM-On-iOS/blob/master/build-llvm.sh
2. https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html#cross-compiling-for-ios-tvos-or-watchos

Unfortunately the following command shows two error messages over and over again.

cmake . -GNinja -B ../iree-build-ios \
-DLLVM_TARGETS_TO_BUILD="AArch64" \
-DLLVM_BUILD_TOOLS=OFF \
-DCLANG_BUILD_TOOLS=OFF \
-DBUILD_SHARED_LIBS=OFF \
-DLLVM_ENABLE_ZLIB=OFF \
-DLLVM_ENABLE_ZSTD=OFF \
-DLLVM_ENABLE_THREADS=OFF \
-DLLVM_ENABLE_UNWIND_TABLES=OFF \
-DLLVM_ENABLE_EH=OFF \
-DLLVM_ENABLE_RTTI=OFF \
-DLLVM_ENABLE_TERMINFO=OFF \
-DLLVM_ENABLE_FFI=ON \
-DLLVM_DISABLE_ASSEMBLY_FILES=ON \
-DFFI_INCLUDE_DIR=$libffiInstallDir/include/ffi \
-DFFI_LIBRARY_DIR=$libffiInstallDir \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=$llvmInstallDir \
-DCMAKE_TOOLCHAIN_FILE=third_party/llvm-project/llvm/cmake/platforms/iOS.cmake \
-DLLVM_TARGET_ARCH=arm64 \
-DCMAKE_OSX_ARCHITECTURES=arm64 \
-DCMAKE_OSX_SYSROOT=$(xcodebuild -version -sdk iphonesimulator Path) \
-DCMAKE_SYSTEM_NAME=iOS \
-DCMAKE_OSX_DEPLOYMENT_TARGET=9.3 \
-DCMAKE_INSTALL_PREFIX=`pwd`/_install \
-DCMAKE_XCODE_ATTRIBUTE_ONLY_ACTIVE_ARCH=NO \
-DCMAKE_IOS_INSTALL_COMBINED=YES > /tmp/o 2>&1

The stdout and stderr combined output is at https://gist.github.com/wangkuiyi/0b37a810e377ab5d182548c7ed7867a9.

Greping shows that all errors came out from two places.

$ grep 'CMake Error' /tmp/o | sort | uniq
CMake Error at build_tools/cmake/flatbuffer_c_library.cmake:91 (add_custom_command):
CMake Error at build_tools/cmake/iree_c_embed_data.cmake:75 (add_custom_command):

More context about these two errors.

CMake Error at build_tools/cmake/flatbuffer_c_library.cmake:91 (add_custom_command):
  Error evaluating generator expression:

    $<TARGET_FILE:iree-flatcc-cli>

  No target "iree-flatcc-cli"
Call Stack (most recent call first):
  runtime/src/iree/base/internal/flatcc/CMakeLists.txt:47 (flatbuffer_c_library)


CMake Error at build_tools/cmake/iree_c_embed_data.cmake:75 (add_custom_command):
  Error evaluating generator expression:

    $<TARGET_FILE:generate_embed_data>

  No target "generate_embed_data"
Call Stack (most recent call first):
  runtime/src/iree/base/testing/CMakeLists.txt:19 (iree_c_embed_data)

I don't know enough about CMake to understand the above errors. Can anyone help?

[ScottTodd]

Hi, thanks for including all of those details!

The IREE project contains both a compiler and a set of runtime libraries. The compiler depends on LLVM and a few other large/complex tools, while the runtime libraries use minimal dependencies. We typically expect that developers will run the compiler on a "host" platform like Linux/Windows/macOS after either building it from source or using one of our binary distributions (e.g. install from Python's pip). After compiling programs ahead of time, we expect that developers will run those compiled programs on a "target" platform like Linux/Windows/macOS/Android/iOS/an embedded device/the web/etc.

The Android docs you linked (https://iree-org.github.io/iree/building-from-source/android/) show how that works in practice for one target platform, configuring CMake for the target build with -DIREE_BUILD_COMPILER=OFF -DIREE_HOST_BINARY_ROOT=.... The same general setup should work for iOS - you'd build the compiler on your host machine (likely a macOS desktop/laptop) then build the runtime for your target machine (an iOS phone/tablet). Note that for the purposes of working on a Metal backend, you might find it easier to stay on a single device (just macOS, no iOS), since cross compiling and managing multiple devices can add extra steps that slow development down.

---

Now, for the specific errors with iree-flatcc-cli and generate_embed_data that you're seeing, those tools are expected to be built on a host platform with a path to the host build provided in -DIREE_HOST_BINARY_ROOT=... and we skip over building them directly when cross compiling:

iree/CMakeLists.txt

Lines 622 to 628 in 663b1a7

	if(NOT CMAKE_CROSSCOMPILING)
	install(
	TARGETS iree-flatcc-cli
	COMPONENT iree-flatcc-cli
	RUNTIME DESTINATION bin
	)
	endif()

iree/build_tools/embed_data/CMakeLists.txt

Lines 7 to 9 in 663b1a7

	if(NOT CMAKE_CROSSCOMPILING)
	add_executable(generate_embed_data)
	target_sources(generate_embed_data PRIVATE generate_embed_data_main.cc)

If you're committed to building the compiler for iOS and not just macOS, you can try removing those if(NOT CMAKE_CROSSCOMPILING) conditions, but I'd suggest against that... or at least discussing your workflow / requirements more before you head down that path. There are scenarios (mainly JIT compilation) where you'd want to build the compiler for a platform like iOS/Android, but the LLVM/MLIR-based compiler is really not designed to make that easy or high performance.

[wangkuiyi]

Thank you, @ScottTodd !

I set "IREE BUILD COMPILER=OFF" to the following.

cmake . -GNinja -B ../iree-build-ios \
  -DIREE_HOST_BINARY_ROOT="$PWD/../iree-build/install" \
  -DIREE_BUILD_COMPILER=OFF \
  -DCMAKE_OSX_SYSROOT=$(xcodebuild -version -sdk iphonesimulator Path) \
  -DCMAKE_SYSTEM_NAME=iOS \
  > /tmp/o 2>&1

There are still errors as shown in the second version of the gist at https://gist.github.com/wangkuiyi/0b37a810e377ab5d182548c7ed7867a9. But now they seem to be about macOS builds.

I deduplicated the errors

$ grep 'CMake Error' /tmp/o  | sort | uniq
CMake Error at build_tools/cmake/iree_cc_binary.cmake:151 (install):
CMake Error at third_party/cpuinfo/CMakeLists.txt:794 (INSTALL):
CMake Error at third_party/cpuinfo/CMakeLists.txt:800 (INSTALL):
CMake Error at third_party/cpuinfo/CMakeLists.txt:806 (INSTALL):

A little bit more context.

CMake Error at third_party/cpuinfo/CMakeLists.txt:794 (INSTALL):
  INSTALL TARGETS given no BUNDLE DESTINATION for MACOSX_BUNDLE executable
  target "isa-info".


CMake Error at third_party/cpuinfo/CMakeLists.txt:800 (INSTALL):
  INSTALL TARGETS given no BUNDLE DESTINATION for MACOSX_BUNDLE executable
  target "cpu-info".


CMake Error at third_party/cpuinfo/CMakeLists.txt:806 (INSTALL):
  INSTALL TARGETS given no BUNDLE DESTINATION for MACOSX_BUNDLE executable
  target "cache-info".


CMake Error at build_tools/cmake/iree_cc_binary.cmake:151 (install):
  install TARGETS given no BUNDLE DESTINATION for MACOSX_BUNDLE executable
  target
  "iree_tests_e2e_models_mnist_fake_weights_llvm_cpu_static_bytecode_test_run".
Call Stack (most recent call first):
  build_tools/cmake/iree_static_linker_test.cmake:235 (iree_cc_binary)
  tests/e2e/models/CMakeLists.txt:91 (iree_static_linker_test)

All of these errors are caused by the call to the install CMake function. I'm not sure if we need to make a MACOS BUNDLE and install the binaries into it. But I am trying to figure out by reading the document of install.

[antiagainst]

Yup. With the above three patches I'm able to cross compile core IREE runtime binaries to iOS.

[antiagainst]

These three patches are all landed now.

[wangkuiyi]

@GMNGeoffrey Thank you for reminding me! I am new to GitHub Discussion. I guess I will have to click "Create issue from discussion" to create an issue for the PRs to refer to.

Thank you @antiagainst for the prompt fixing! Amazingly fast and awesome!

[GMNGeoffrey]

Ehh it's ok, but I think next time for this sort of thing it's fine to file an issue directly. The PRs and commits will still link to this and there's now a link from this to the PRs. GitHub doesn't really have any more structured cross-linking

[wangkuiyi]

Yep. I agree.

[antiagainst]

@wangkuiyi: @ScottTodd did a great job to explain how the compilation flow should work in general. Echoing his point that normally we don't need to cross-compile and run the compilers on the target device (iOS here); they are meant to be invoked on the host device (macOS) to compile the source model into deployable formats (IREE vmfb files) that contains both the host scheduling logic and device native kernels (dylib for CPU, SPIR-V for Vulkan, etc.).

I tried out iOS cross compilation on my side; there are a few small issues to be fixed to make it work actually, which I sent out separate pull requests for them. With them, the following should work:

# Building IREE on host macOS machines (in IREE root directory):
cmake -S . -B build-release/ -GNinja \
  -DCMAKE_BUILD_TYPE=RelWithDebInfo
  -DCMAKE_INSTALL_PREFIX=build-release/install
cmake --build build-release/ --target install

# Cross compiling to target iOS device (in IREE root directory):
cmake -S . -B build-ios -GNinja 
  -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_OSX_ARCHITECTURES=arm64 \
  -DCMAKE_SYSTEM_PROCESSOR=arm64 \
  -DCMAKE_OSX_DEPLOYMENT_TARGET=11.0 \
  -DCMAKE_IOS_INSTALL_COMBINED=YES \
  -DIREE_HOST_BINARY_ROOT=$PWD/build-release/install \
  -DCMAKE_INSTALL_PREFIX=build-ios/install \
  -DIREE_BUILD_COMPILER=OFF -DIREE_BUILD_TESTS=OFF -DIREE_BUILD_SAMPLES=OFF
cmake --build build-ios --config Release --target install

Though I don't have an iPhone at my side so cannot verify it works properly. Also those binaries built are meant to be invoked from command-line as they are developer tools effectively. Proper mobile apps should be linking/calling into IREE libraries directly.

[wangkuiyi]

Thank you @antiagainst so much! I confirm it works now.