Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Executorch Vulkan delegate fails to build vulkan_executor_runner without NDK #7343

Open
alexggener opened this issue Dec 17, 2024 · 7 comments
Open

Executorch Vulkan delegate fails to build vulkan_executor_runner without NDK #7343

alexggener opened this issue Dec 17, 2024 · 7 comments
Assignees
@SS-JIA
Labels
module: vulkan triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@alexggener
Copy link

🐛 Describe the bug

Trying to use executorch Vulkan delegate 0.4.0 on a Jetson Orin with Vulkan installed.

Successfully installed executorch and tested without Vulkan delegate.
Configured CMAKE as:

(rm -rf cmake-vk-out && mkdir cmake-vk-out && cd cmake-vk-out && cmake .. -DEXECUTORCH_BUILD_VULKAN=ON)

CMAKE summary config is:

-- ******** Summary ********
--   CMAKE_BUILD_TYPE              : Debug
--   CMAKE_CXX_STANDARD            : 17
--   CMAKE_CXX_COMPILER_ID         : GNU
--   CMAKE_TOOLCHAIN_FILE          : 
--   BUCK2                         : /home/jetson/alex/executorch/cmake-vk-out/buck2-bin/buck2-49670bee56a7d8a7696409ca6fbf7551d2469787
--   PYTHON_EXECUTABLE             : python3
--   FLATC_EXECUTABLE              : flatc
--   EXECUTORCH_ENABLE_LOGGING              : ON
--   EXECUTORCH_ENABLE_PROGRAM_VERIFICATION : ON
--   EXECUTORCH_LOG_LEVEL                   : Info
--   EXECUTORCH_BUILD_ANDROID_JNI           : OFF
--   EXECUTORCH_BUILD_ARM_BAREMETAL         : OFF
--   EXECUTORCH_BUILD_COREML                : OFF
--   EXECUTORCH_BUILD_KERNELS_CUSTOM        : OFF
--   EXECUTORCH_BUILD_EXECUTOR_RUNNER       : ON
--   EXECUTORCH_BUILD_EXTENSION_DATA_LOADER : OFF
--   EXECUTORCH_BUILD_EXTENSION_MODULE      : OFF
--   EXECUTORCH_BUILD_EXTENSION_RUNNER_UTIL : OFF
--   EXECUTORCH_BUILD_EXTENSION_TENSOR      : OFF
--   EXECUTORCH_BUILD_FLATC                 : ON
--   EXECUTORCH_BUILD_GFLAGS                : ON
--   EXECUTORCH_BUILD_GTESTS                : OFF
--   EXECUTORCH_BUILD_HOST_TARGETS          : ON
--   EXECUTORCH_BUILD_MPS                   : OFF
--   EXECUTORCH_BUILD_PYBIND                : OFF
--   EXECUTORCH_BUILD_QNN                   : OFF
--   EXECUTORCH_BUILD_KERNELS_OPTIMIZED     : OFF
--   EXECUTORCH_BUILD_KERNELS_QUANTIZED     : OFF
--   EXECUTORCH_BUILD_DEVTOOLS              : OFF
--   EXECUTORCH_BUILD_SIZE_TEST             : OFF
--   EXECUTORCH_BUILD_XNNPACK               : OFF
--   EXECUTORCH_BUILD_VULKAN                : ON
--   EXECUTORCH_BUILD_PTHREADPOOL           : ON
--   EXECUTORCH_BUILD_CPUINFO               : ON

But failed when building the vulkan_executor_runner. See below the verbose log output of the error:

[ 23%] Building CXX object kernels/portable/CMakeFiles/portable_kernels.dir/cpu/op_clamp.cpp.o
cd /home/jetson/alex/executorch/cmake-vk-out/kernels/portable && /usr/bin/c++ -DET_MIN_LOG_LEVEL=Info -I/home/jetson/alex/executorch/.. -std=c++17 -O0 -g -std=gnu++17 -Wno-deprecated-declarations -fPIC -MD -MT kernels/portable/CMakeFiles/portable_kernels.dir/cpu/op_clamp.cpp.o -MF CMakeFiles/portable_kernels.dir/cpu/op_clamp.cpp.o.d -o CMakeFiles/portable_kernels.dir/cpu/op_clamp.cpp.o -c /home/jetson/alex/executorch/kernels/portable/cpu/op_clamp.cpp
In file included from /home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:9:
/home/jetson/alex/executorch/backends/vulkan/../../../executorch/backends/vulkan/runtime/graph/containers/Types.h:22:6: warning: elaborated-type-specifier for a scoped enum must not use the ‘class’ keyword
   22 | enum class TypeTag : uint32_t {
      | ~~~~ ^~~~~
      |      -----
/home/jetson/alex/executorch/backends/vulkan/../../../executorch/backends/vulkan/runtime/graph/containers/Types.h:22:20: error: found ‘:’ in nested-name-specifier, expected ‘::’
   22 | enum class TypeTag : uint32_t {
      |                    ^
      |                    ::
/home/jetson/alex/executorch/backends/vulkan/../../../executorch/backends/vulkan/runtime/graph/containers/Types.h:22:12: error: ‘TypeTag’ has not been declared
   22 | enum class TypeTag : uint32_t {
      |            ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/../../../executorch/backends/vulkan/runtime/graph/containers/Types.h:22:31: error: expected unqualified-id before ‘{’ token
   22 | enum class TypeTag : uint32_t {
      |                               ^
/home/jetson/alex/executorch/backends/vulkan/../../../executorch/backends/vulkan/runtime/graph/containers/Types.h:42:51: error: ‘TypeTag’ does not name a type
   42 | std::ostream& operator<<(std::ostream& out, const TypeTag& tag);
      |                                                   ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:18:51: error: ‘TypeTag’ does not name a type
   18 | std::ostream& operator<<(std::ostream& out, const TypeTag& tag) {
      |                                                   ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp: In function ‘std::ostream& vkcompute::operator<<(std::ostream&, const int&)’:
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:20:5: note: in expansion of macro ‘PRINT_CASE’
   20 |     PRINT_CASE(NONE)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:21:5: note: in expansion of macro ‘PRINT_CASE’
   21 |     PRINT_CASE(INT)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:22:5: note: in expansion of macro ‘PRINT_CASE’
   22 |     PRINT_CASE(DOUBLE)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:23:5: note: in expansion of macro ‘PRINT_CASE’
   23 |     PRINT_CASE(BOOL)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:24:5: note: in expansion of macro ‘PRINT_CASE’
   24 |     PRINT_CASE(TENSOR)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:25:5: note: in expansion of macro ‘PRINT_CASE’
   25 |     PRINT_CASE(STAGING)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:26:5: note: in expansion of macro ‘PRINT_CASE’
   26 |     PRINT_CASE(TENSORREF)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:27:5: note: in expansion of macro ‘PRINT_CASE’
   27 |     PRINT_CASE(INTLIST)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:28:5: note: in expansion of macro ‘PRINT_CASE’
   28 |     PRINT_CASE(DOUBLELIST)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:29:5: note: in expansion of macro ‘PRINT_CASE’
   29 |     PRINT_CASE(BOOLLIST)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:30:5: note: in expansion of macro ‘PRINT_CASE’
   30 |     PRINT_CASE(VALUELIST)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:31:5: note: in expansion of macro ‘PRINT_CASE’
   31 |     PRINT_CASE(STRING)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:32:5: note: in expansion of macro ‘PRINT_CASE’
   32 |     PRINT_CASE(SYMINT)
      |     ^~~~~~~~~~
gmake[3]: *** [backends/vulkan/CMakeFiles/vulkan_backend.dir/build.make:191: backends/vulkan/CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.o] Error 1

Seems like C++17 standard was not well configured, but it can be seen in the command that the flags -std=c++17 -O0 -g -std=gnu++17 are set.

Extra information about the device:
Ubuntu 22.04
Jetpack 6.1
GCC-13.1
Python 3.10
Vulkan 1.3.204
glslc info:
shaderc 2023.2-1
spirv-tools 2022.4+1.3.236.0-1
glslang 11.13.0-1
Target: SPIR-V 1.0

Versions

Error in cpuinfo: prctl(PR_SVE_GET_VL) failed
Collecting environment information...
PyTorch version: 2.5.0
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.5 LTS (aarch64)
GCC version: (Ubuntu 13.1.0-8ubuntu1~22.04) 13.1.0
Clang version: Could not collect
CMake version: version 3.31.2
Libc version: glibc-2.35

Python version: 3.10.12 (main, Nov  6 2024, 20:22:13) [GCC 11.4.0] (64-bit runtime)
Python platform: Linux-5.15.148-tegra-aarch64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: 12.2.140
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: GPU 0: Orin (nvgpu)
Nvidia driver version: 540.4.0
cuDNN version: Probably one of the following:
/usr/lib/aarch64-linux-gnu/libcudnn.so.8.9.4
/usr/lib/aarch64-linux-gnu/libcudnn.so.9.3.0
/usr/lib/aarch64-linux-gnu/libcudnn_adv.so.9.3.0
/usr/lib/aarch64-linux-gnu/libcudnn_adv_infer.so.8.9.4
/usr/lib/aarch64-linux-gnu/libcudnn_adv_train.so.8.9.4
/usr/lib/aarch64-linux-gnu/libcudnn_cnn.so.9.3.0
/usr/lib/aarch64-linux-gnu/libcudnn_cnn_infer.so.8.9.4
/usr/lib/aarch64-linux-gnu/libcudnn_cnn_train.so.8.9.4
/usr/lib/aarch64-linux-gnu/libcudnn_engines_precompiled.so.9.3.0
/usr/lib/aarch64-linux-gnu/libcudnn_engines_runtime_compiled.so.9.3.0
/usr/lib/aarch64-linux-gnu/libcudnn_graph.so.9.3.0
/usr/lib/aarch64-linux-gnu/libcudnn_heuristic.so.9.3.0
/usr/lib/aarch64-linux-gnu/libcudnn_ops.so.9.3.0
/usr/lib/aarch64-linux-gnu/libcudnn_ops_infer.so.8.9.4
/usr/lib/aarch64-linux-gnu/libcudnn_ops_train.so.8.9.4
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                       aarch64
CPU op-mode(s):                     32-bit, 64-bit
Byte Order:                         Little Endian
CPU(s):                             8
On-line CPU(s) list:                0-3
Off-line CPU(s) list:               4-7
Vendor ID:                          ARM
Model name:                         Cortex-A78AE
Model:                              1
Thread(s) per core:                 1
Core(s) per cluster:                4
Socket(s):                          -
Cluster(s):                         1
Stepping:                           r0p1
CPU max MHz:                        1984.0000
CPU min MHz:                        115.2000
BogoMIPS:                           62.50
Flags:                              fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp uscat ilrcpc flagm paca pacg
L1d cache:                          256 KiB (4 instances)
L1i cache:                          256 KiB (4 instances)
L2 cache:                           1 MiB (4 instances)
L3 cache:                           2 MiB (1 instance)
NUMA node(s):                       1
NUMA node0 CPU(s):                  0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:           Mitigation; __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; CSV2, but not BHB
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Versions of relevant libraries:
[pip3] executorch==0.4.0a0+6a085ff
[pip3] numpy==1.21.3
[pip3] torch==2.5.0
[pip3] torchaudio==2.5.0
[pip3] torchsr==1.0.4
[pip3] torchvision==0.20.0
[conda] Could not collect
@alexggener
Copy link
Author

Edit. Apologize, the log error below is the correct one. It fails on the Building CXX object backends/vulkan/CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.o

[ 20%] Building CXX object backends/vulkan/CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.o
cd /home/jetson/alex/executorch/cmake-vk-out/backends/vulkan && /usr/bin/c++ -DET_MIN_LOG_LEVEL=Info -I/home/jetson/alex/executorch/cmake-vk-out/schema/include -I/home/jetson/alex/executorch/backends/vulkan/../../.. -I/home/jetson/alex/executorch/backends/vulkan/third-party/Vulkan-Headers/include -I/home/jetson/alex/executorch/backends/vulkan/third-party/volk -I/home/jetson/alex/executorch/backends/vulkan/third-party/VulkanMemoryAllocator -I/home/jetson/alex/executorch/backends/vulkan/../../third-party/flatbuffers/include -I/home/jetson/alex/executorch/.. -std=c++17 -O0 -g -std=gnu++17 -fexceptions -DUSE_VULKAN_WRAPPER -DUSE_VULKAN_VOLK -Wno-deprecated-declarations -fPIC -MD -MT backends/vulkan/CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.o -MF CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.o.d -o CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.o -c /home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp
In file included from /home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:9:
/home/jetson/alex/executorch/backends/vulkan/../../../executorch/backends/vulkan/runtime/graph/containers/Types.h:22:6: warning: elaborated-type-specifier for a scoped enum must not use the ‘class’ keyword
   22 | enum class TypeTag : uint32_t {
      | ~~~~ ^~~~~
      |      -----
/home/jetson/alex/executorch/backends/vulkan/../../../executorch/backends/vulkan/runtime/graph/containers/Types.h:22:20: error: found ‘:’ in nested-name-specifier, expected ‘::’
   22 | enum class TypeTag : uint32_t {
      |                    ^
      |                    ::
/home/jetson/alex/executorch/backends/vulkan/../../../executorch/backends/vulkan/runtime/graph/containers/Types.h:22:12: error: ‘TypeTag’ has not been declared
   22 | enum class TypeTag : uint32_t {
      |            ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/../../../executorch/backends/vulkan/runtime/graph/containers/Types.h:22:31: error: expected unqualified-id before ‘{’ token
   22 | enum class TypeTag : uint32_t {
      |                               ^
/home/jetson/alex/executorch/backends/vulkan/../../../executorch/backends/vulkan/runtime/graph/containers/Types.h:42:51: error: ‘TypeTag’ does not name a type
   42 | std::ostream& operator<<(std::ostream& out, const TypeTag& tag);
      |                                                   ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:18:51: error: ‘TypeTag’ does not name a type
   18 | std::ostream& operator<<(std::ostream& out, const TypeTag& tag) {
      |                                                   ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp: In function ‘std::ostream& vkcompute::operator<<(std::ostream&, const int&)’:
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:20:5: note: in expansion of macro ‘PRINT_CASE’
   20 |     PRINT_CASE(NONE)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:21:5: note: in expansion of macro ‘PRINT_CASE’
   21 |     PRINT_CASE(INT)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:22:5: note: in expansion of macro ‘PRINT_CASE’
   22 |     PRINT_CASE(DOUBLE)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:23:5: note: in expansion of macro ‘PRINT_CASE’
   23 |     PRINT_CASE(BOOL)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:24:5: note: in expansion of macro ‘PRINT_CASE’
   24 |     PRINT_CASE(TENSOR)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:25:5: note: in expansion of macro ‘PRINT_CASE’
   25 |     PRINT_CASE(STAGING)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:26:5: note: in expansion of macro ‘PRINT_CASE’
   26 |     PRINT_CASE(TENSORREF)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:27:5: note: in expansion of macro ‘PRINT_CASE’
   27 |     PRINT_CASE(INTLIST)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:28:5: note: in expansion of macro ‘PRINT_CASE’
   28 |     PRINT_CASE(DOUBLELIST)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:29:5: note: in expansion of macro ‘PRINT_CASE’
   29 |     PRINT_CASE(BOOLLIST)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:30:5: note: in expansion of macro ‘PRINT_CASE’
   30 |     PRINT_CASE(VALUELIST)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:31:5: note: in expansion of macro ‘PRINT_CASE’
   31 |     PRINT_CASE(STRING)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:32:5: note: in expansion of macro ‘PRINT_CASE’
   32 |     PRINT_CASE(SYMINT)
      |     ^~~~~~~~~~
gmake[3]: *** [backends/vulkan/CMakeFiles/vulkan_backend.dir/build.make:191: backends/vulkan/CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.o] Error 1
gmake[3]: Leaving directory '/home/jetson/alex/executorch/cmake-vk-out'
gmake[2]: *** [CMakeFiles/Makefile2:792: backends/vulkan/CMakeFiles/vulkan_backend.dir/all] Error 2
gmake[2]: Leaving directory '/home/jetson/alex/executorch/cmake-vk-out'
gmake[1]: *** [CMakeFiles/Makefile2:837: backends/vulkan/CMakeFiles/vulkan_executor_runner.dir/rule] Error 2
gmake[1]: Leaving directory '/home/jetson/alex/executorch/cmake-vk-out'
gmake: *** [Makefile:364: vulkan_executor_runner] Error 2

@alexggener
Copy link
Author

I was able to run the compilation command with verbose, to give more visibility to the problem. See below the log output:

(executorch_env) jetson@ubuntu:~/alex/executorch/cmake-vk-out/backends/vulkan$ cd /home/jetson/alex/executorch/cmake-vk-out/backends/vulkan && /usr/bin/c++ -DET_MIN_LOG_LEVEL=Info -I/home/jetson/alex/executorch/cmake-vk-out/schema/include -I/home/jetson/alex/executorch/backends/vulkan/../../.. -I/home/jetson/alex/executorch/backends/vulkan/third-party/Vulkan-Headers/include -I/home/jetson/alex/executorch/backends/vulkan/third-party/volk -I/home/jetson/alex/executorch/backends/vulkan/third-party/VulkanMemoryAllocator -I/home/jetson/alex/executorch/backends/vulkan/../../third-party/flatbuffers/include -I/home/jetson/alex/executorch/.. -std=c++17 -O0 -g -std=gnu++17 -fexceptions -DUSE_VULKAN_WRAPPER -DUSE_VULKAN_VOLK -Wno-deprecated-declarations -fPIC -MD -MT backends/vulkan/CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.o -MF CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.o.d -o CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.o -c /home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp --verbose
Using built-in specs.
COLLECT_GCC=/usr/bin/c++
Target: aarch64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 13.1.0-8ubuntu1~22.04' --with-bugurl=file:///usr/share/doc/gcc-13/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2,rust --prefix=/usr --with-gcc-major-version-only --program-suffix=-13 --program-prefix=aarch64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/libexec --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libquadmath --disable-libquadmath-support --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --enable-fix-cortex-a53-843419 --disable-werror --enable-checking=release --build=aarch64-linux-gnu --host=aarch64-linux-gnu --target=aarch64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-serialization=2
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 13.1.0 (Ubuntu 13.1.0-8ubuntu1~22.04) 
COLLECT_GCC_OPTIONS='-D' 'ET_MIN_LOG_LEVEL=Info' '-I' '/home/jetson/alex/executorch/cmake-vk-out/schema/include' '-I' '/home/jetson/alex/executorch/backends/vulkan/../../..' '-I' '/home/jetson/alex/executorch/backends/vulkan/third-party/Vulkan-Headers/include' '-I' '/home/jetson/alex/executorch/backends/vulkan/third-party/volk' '-I' '/home/jetson/alex/executorch/backends/vulkan/third-party/VulkanMemoryAllocator' '-I' '/home/jetson/alex/executorch/backends/vulkan/../../third-party/flatbuffers/include' '-I' '/home/jetson/alex/executorch/..' '-std=c++17' '-O0' '-g' '-std=gnu++17' '-fexceptions' '-D' 'USE_VULKAN_WRAPPER' '-D' 'USE_VULKAN_VOLK' '-Wno-deprecated-declarations' '-fPIC' '-MD' '-MT' 'backends/vulkan/CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.o' '-MF' 'CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.o.d' '-o' 'CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.o' '-c' '-v' '-shared-libgcc' '-mlittle-endian' '-mabi=lp64' '-dumpdir' 'CMakeFiles/vulkan_backend.dir/runtime/graph/containers/'
 /usr/libexec/gcc/aarch64-linux-gnu/13/cc1plus -quiet -v -I /home/jetson/alex/executorch/cmake-vk-out/schema/include -I /home/jetson/alex/executorch/backends/vulkan/../../.. -I /home/jetson/alex/executorch/backends/vulkan/third-party/Vulkan-Headers/include -I /home/jetson/alex/executorch/backends/vulkan/third-party/volk -I /home/jetson/alex/executorch/backends/vulkan/third-party/VulkanMemoryAllocator -I /home/jetson/alex/executorch/backends/vulkan/../../third-party/flatbuffers/include -I /home/jetson/alex/executorch/.. -imultiarch aarch64-linux-gnu -MD CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.d -MF CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.o.d -MT backends/vulkan/CMakeFiles/vulkan_backend.dir/runtime/graph/containers/Types.cpp.o -D_GNU_SOURCE -D ET_MIN_LOG_LEVEL=Info -D USE_VULKAN_WRAPPER -D USE_VULKAN_VOLK /home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp -quiet -dumpdir CMakeFiles/vulkan_backend.dir/runtime/graph/containers/ -dumpbase Types.cpp.cpp -dumpbase-ext .cpp -mlittle-endian -mabi=lp64 -g -O0 -Wno-deprecated-declarations -std=c++17 -std=gnu++17 -version -fexceptions -fPIC -fasynchronous-unwind-tables -fstack-protector-strong -Wformat -Wformat-security -fstack-clash-protection -o /tmp/ccVrXrIz.s
GNU C++17 (Ubuntu 13.1.0-8ubuntu1~22.04) version 13.1.0 (aarch64-linux-gnu)
        compiled by GNU C version 13.1.0, GMP version 6.2.1, MPFR version 4.1.0, MPC version 1.2.1, isl version isl-0.24-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring duplicate directory "/usr/include/aarch64-linux-gnu/c++/13"
ignoring nonexistent directory "/usr/local/include/aarch64-linux-gnu"
ignoring nonexistent directory "/usr/lib/gcc/aarch64-linux-gnu/13/include-fixed/aarch64-linux-gnu"
ignoring nonexistent directory "/usr/lib/gcc/aarch64-linux-gnu/13/include-fixed"
ignoring nonexistent directory "/usr/lib/gcc/aarch64-linux-gnu/13/../../../../aarch64-linux-gnu/include"
ignoring duplicate directory "/home/jetson/alex/executorch/.."
#include "..." search starts here:
#include <...> search starts here:
 /home/jetson/alex/executorch/cmake-vk-out/schema/include
 /home/jetson/alex/executorch/backends/vulkan/../../..
 /home/jetson/alex/executorch/backends/vulkan/third-party/Vulkan-Headers/include
 /home/jetson/alex/executorch/backends/vulkan/third-party/volk
 /home/jetson/alex/executorch/backends/vulkan/third-party/VulkanMemoryAllocator
 /home/jetson/alex/executorch/backends/vulkan/../../third-party/flatbuffers/include
 /usr/include/c++/13
 /usr/include/aarch64-linux-gnu/c++/13
 /usr/include/c++/13/backward
 /usr/lib/gcc/aarch64-linux-gnu/13/include
 /usr/local/include
 /usr/include/aarch64-linux-gnu
 /usr/include
End of search list.
Compiler executable checksum: 373057ccc7d49116cd5e77a8e3db8c2d
In file included from /home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:9:
/home/jetson/alex/executorch/backends/vulkan/../../../executorch/backends/vulkan/runtime/graph/containers/Types.h:22:6: warning: elaborated-type-specifier for a scoped enum must not use the ‘class’ keyword
   22 | enum class TypeTag : uint32_t {
      | ~~~~ ^~~~~
      |      -----
/home/jetson/alex/executorch/backends/vulkan/../../../executorch/backends/vulkan/runtime/graph/containers/Types.h:22:20: error: found ‘:’ in nested-name-specifier, expected ‘::’
   22 | enum class TypeTag : uint32_t {
      |                    ^
      |                    ::
/home/jetson/alex/executorch/backends/vulkan/../../../executorch/backends/vulkan/runtime/graph/containers/Types.h:22:12: error: ‘TypeTag’ has not been declared
   22 | enum class TypeTag : uint32_t {
      |            ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/../../../executorch/backends/vulkan/runtime/graph/containers/Types.h:22:31: error: expected unqualified-id before ‘{’ token
   22 | enum class TypeTag : uint32_t {
      |                               ^
/home/jetson/alex/executorch/backends/vulkan/../../../executorch/backends/vulkan/runtime/graph/containers/Types.h:42:51: error: ‘TypeTag’ does not name a type
   42 | std::ostream& operator<<(std::ostream& out, const TypeTag& tag);
      |                                                   ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:18:51: error: ‘TypeTag’ does not name a type
   18 | std::ostream& operator<<(std::ostream& out, const TypeTag& tag) {
      |                                                   ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp: In function ‘std::ostream& vkcompute::operator<<(std::ostream&, const int&)’:
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:20:5: note: in expansion of macro ‘PRINT_CASE’
   20 |     PRINT_CASE(NONE)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:21:5: note: in expansion of macro ‘PRINT_CASE’
   21 |     PRINT_CASE(INT)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:22:5: note: in expansion of macro ‘PRINT_CASE’
   22 |     PRINT_CASE(DOUBLE)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:23:5: note: in expansion of macro ‘PRINT_CASE’
   23 |     PRINT_CASE(BOOL)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:24:5: note: in expansion of macro ‘PRINT_CASE’
   24 |     PRINT_CASE(TENSOR)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:25:5: note: in expansion of macro ‘PRINT_CASE’
   25 |     PRINT_CASE(STAGING)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:26:5: note: in expansion of macro ‘PRINT_CASE’
   26 |     PRINT_CASE(TENSORREF)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:27:5: note: in expansion of macro ‘PRINT_CASE’
   27 |     PRINT_CASE(INTLIST)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:28:5: note: in expansion of macro ‘PRINT_CASE’
   28 |     PRINT_CASE(DOUBLELIST)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:29:5: note: in expansion of macro ‘PRINT_CASE’
   29 |     PRINT_CASE(BOOLLIST)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:30:5: note: in expansion of macro ‘PRINT_CASE’
   30 |     PRINT_CASE(VALUELIST)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:31:5: note: in expansion of macro ‘PRINT_CASE’
   31 |     PRINT_CASE(STRING)
      |     ^~~~~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:14:8: error: ‘TypeTag’ has not been declared
   14 |   case TypeTag::name:    \
      |        ^~~~~~~
/home/jetson/alex/executorch/backends/vulkan/runtime/graph/containers/Types.cpp:32:5: note: in expansion of macro ‘PRINT_CASE’
   32 |     PRINT_CASE(SYMINT)
      |     ^~~~~~~~~~

@Jack-Khuu Jack-Khuu added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Dec 17, 2024
@Jack-Khuu
Copy link
Contributor

cc @SS-JIA

@SS-JIA
Copy link
Contributor

SS-JIA commented Dec 18, 2024 •
edited
Loading

@alexggener Thanks for raising the issue!

First, I would encourage you to try out building from master; there have been a lot of updates to the Vulkan backend since the 0.4 release and it's the branch I use to develop against so it will be a bit easier for me to assist.

Just to establish a baseline, can you try to build and run the Vulkan backend from the current master branch with the following commands? For reference, I am able to get the Vulkan backend working with these commands on a CentOS8 machine with a NVIDIA A100 GPU.

# 1. Clean your workspace and install dependencies, just for good luck
./install_requirements.sh --clean
git submodule sync
git submodule update --init

./install_requirements.sh --pybind

# 2. Build the ExecuTorch runtime and the Vulkan backend
(rm -rf cmake-out && \
  pp cmake . \
  -DCMAKE_INSTALL_PREFIX=cmake-out \
  -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
  -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
  -DEXECUTORCH_BUILD_EXTENSION_RUNNER_UTIL=ON \
  -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \
  -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON \
  -DEXECUTORCH_BUILD_KERNELS_CUSTOM=ON \
  -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON \
  -DEXECUTORCH_BUILD_TESTS=ON \
  -DEXECUTORCH_BUILD_VULKAN=ON \
  -Bcmake-out && \
  cmake --build cmake-out -j16 --target install)

# 3. Build the Vulkan test binary
(rm -rf cmake-out/backends/vulkan/test && \
  pp cmake backends/vulkan/test \
    -DCMAKE_INSTALL_PREFIX=cmake-out \
    -DPYTHON_EXECUTABLE=python \
    -Bcmake-out/backends/vulkan/test && \
  cmake --build cmake-out/backends/vulkan/test -j16)

# 4. Run the test binary
cmake-out/backends/vulkan/test/vulkan_compute_api_test --gtest_filter="*print*"

You should see some output like the below:

[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from VulkanComputeAPITest
[ RUN      ] VulkanComputeAPITest.print_adapter
{
  Physical Device Info {
    apiVersion:    1.3
    driverversion: 2308342208
    deviceType:    DISCRETE_GPU
    deviceName:    NVIDIA PG509-210
    Physical Device Limits {
      maxImageDimension1D                 32768
      maxImageDimension2D                 32768
      maxImageDimension3D                 16384
      maxTexelBufferElements              134217728
      maxPushConstantsSize                256
      maxMemoryAllocationCount            4294967295
      maxSamplerAllocationCount           4000
      maxComputeSharedMemorySize          49152
      maxComputeWorkGroupCount            2147483647,65535,65535
      maxComputeWorkGroupInvocations      1024
      maxComputeWorkGroupSize             1024,1024,64
    }
    16bit Storage Features {
      storageBuffer16BitAccess            1
      uniformAndStorageBuffer16BitAccess  1
      storagePushConstant16               1
      storageInputOutput16                0
    }
    8bit Storage Features {
      storageBuffer8BitAccess             1
      uniformAndStorageBuffer8BitAccess   1
      storagePushConstant8                1
    }
    Shader 16bit and 8bit Features {
      shaderFloat16                       1
      shaderInt8                          1
    }
  }
  Memory Info {
    Memory Types [
       [Heap 1] |
       [Heap 0]  DEVICE_LOCAL |
       [Heap 0]  DEVICE_LOCAL |
       [Heap 1]  HOST_VISIBLE | HOST_COHERENT |
       [Heap 1]  HOST_VISIBLE | HOST_COHERENT | HOST_CACHED |
       [Heap 0]  DEVICE_LOCAL | HOST_VISIBLE | HOST_COHERENT |
    ]
    Memory Heaps [
      85899345920
      142186447872
    ]
  }
  Queue Families {
    (16 Queues)  GRAPHICS | COMPUTE | TRANSFER |
    (2 Queues)  TRANSFER |
    (8 Queues)  COMPUTE | TRANSFER |
    (5 Queues)  TRANSFER |
    (1 Queues)  TRANSFER |
  }
  VkDevice: 0xc97f20
  Compute Queues [
    Family 0, Queue 0: 0xc5a330
  ]
}

[       OK ] VulkanComputeAPITest.print_adapter (393 ms)
[ RUN      ] VulkanComputeAPITest.print_object_sizes
vTensor size: 544 B
Value size: 48 B
StagingBuffer size: 96 B
ComputeGraph size: 384 B
DispatchNode size: 312 B
[       OK ] VulkanComputeAPITest.print_object_sizes (0 ms)
[----------] 2 tests from VulkanComputeAPITest (393 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test suite ran. (393 ms total)
[  PASSED  ] 2 tests.

I was able to test these instructions on the current master branch, on the following commit:

Wed Dec 18 12:14:34 2024 -0600 6ab43991c (main) [ET-VK] Replace Uniform buffers with push constants for permute op (#7347)  [pytorchbot]

@alexggener
Copy link
Author

Hi @SS-JIA,

Thanks a lot for your fast response. I was able to update to the master branch. However, same error appeared. I was able to fix it adding the cstdint library to executorch/backends/vulkan/runtime/graph/containers/Types.h

#pragma once

#include <ostream>
#include <cstdint>

namespace vkcompute {
...

Also, other things to consider are:

  • In the commands you mentioned, there is a pp command. The only reference I've found is the perl-packager, which does not work with the cmake command. I understood it as a typo and removed it, but please, correct me if I'm wrong.
  • When building executorch, on the install_requirements.py in the master branch, there is an incompatibility between torch and torch_vision on the Nightly version. See below
  ERROR: Cannot install torch==2.6.0.dev20241112 and torchvision==0.20.0.dev20241112 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested torch==2.6.0.dev20241112
    torchvision 0.20.0.dev20241112 depends on torch==2.6.0.dev20241111

I've fixed it changing the NIGHTLY_VERSION to dev20241111 and calling for "torchvision==0.20.0.dev20241112" but maybe it is something to consider for further versions.

Thanks for your help and let me know if you want me to open these issues in another report.

Alex.

@SS-JIA
Copy link
Contributor

SS-JIA commented Dec 20, 2024

@alexggener

In the commands you mentioned, there is a pp command. The only reference I've found is the perl-packager, which does not work with the cmake command. I understood it as a typo and removed it, but please, correct me if I'm wrong.

Ah, sorry about that :p this is a quirk of my development machine which must access the internet via a proxy. You are correct in removing it.

When building executorch, on the install_requirements.py in the master branch, there is an incompatibility between torch and torch_vision on the Nightly version.

Interesting, this is somewhat unexpected. It could have arisen due to torchvision changing the requirements of their nightly version recently, as I would have expected to encounter this error when I tried to reproduce. In any case, #7273 was landed recently which updates the pinned commits of these dependencies so hopefully this issue will be fixed if you fast forward.

I was able to fix it adding the cstdint library to executorch/backends/vulkan/runtime/graph/containers/Types.h

Once you added that include, was the binary able to build and run successfully? Just want to confirm that it fixes the issue; I can put up a PR to add the include to the file.

@alexggener
Copy link
Author

@SS-JIA

Yes. Binaries worked perfectly, see below the output of the vulkan_compute_api_test

(executorch_env) jetson@ubuntu:~/alex/executorch$ cmake-out/backends/vulkan/test/vulkan_compute_api_test --gtest_filter="*print*"
Running main() from /home/jetson/alex/executorch/third-party/googletest/googletest/src/gtest_main.cc
Note: Google Test filter = *print*
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from VulkanComputeAPITest
[ RUN      ] VulkanComputeAPITest.print_adapter
{
  Physical Device Info {
    apiVersion:    1.3
    driverversion: 2264989696
    deviceType:    INTEGRATED_GPU
    deviceName:    NVIDIA Tegra Orin (nvgpu)
    Physical Device Limits {
      maxImageDimension1D                 32768
      maxImageDimension2D                 32768
      maxImageDimension3D                 16384
      maxTexelBufferElements              134217728
      maxPushConstantsSize                256
      maxMemoryAllocationCount            4294967295
      maxSamplerAllocationCount           4000
      maxComputeSharedMemorySize          49152
      maxComputeWorkGroupCount            2147483647,65535,65535
      maxComputeWorkGroupInvocations      1024
      maxComputeWorkGroupSize             1024,1024,64
    }
    16bit Storage Features {
      storageBuffer16BitAccess            1
      uniformAndStorageBuffer16BitAccess  1
      storagePushConstant16               1
      storageInputOutput16                0
    }
    8bit Storage Features {
      storageBuffer8BitAccess             1
      uniformAndStorageBuffer8BitAccess   1
      storagePushConstant8                1
    }
    Shader 16bit and 8bit Features {
      shaderFloat16                       1
      shaderInt8                          1
    }
  }
  Memory Info {
    Memory Types [
       [Heap 0]  DEVICE_LOCAL |
       [Heap 0]  DEVICE_LOCAL | HOST_VISIBLE | HOST_COHERENT |
       [Heap 0]  DEVICE_LOCAL | HOST_VISIBLE | HOST_COHERENT | HOST_CACHED |
    ]
    Memory Heaps [
      16382648320
    ]
  }
  Queue Families {
    (16 Queues)  GRAPHICS | COMPUTE | TRANSFER |
    (2 Queues)  TRANSFER |
    (8 Queues)  COMPUTE | TRANSFER |
  }
  VkDevice: 0xaaaad700f880
  Compute Queues [
    Family 0, Queue 0: 0xaaaad6f1ca00
  ]
}

[       OK ] VulkanComputeAPITest.print_adapter (137 ms)
[ RUN      ] VulkanComputeAPITest.print_object_sizes
vTensor size: 544 B
Value size: 48 B
StagingBuffer size: 96 B
ComputeGraph size: 384 B
DispatchNode size: 312 B
[       OK ] VulkanComputeAPITest.print_object_sizes (0 ms)
[----------] 2 tests from VulkanComputeAPITest (137 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test suite ran. (137 ms total)
[  PASSED  ] 2 tests.

I've also tested a quick add example with vulkan_executor_runner and worked as well. See output below.

(executorch_env) jetson@ubuntu:~/alex/executorch$ ./cmake-out/backends/vulkan/vulkan_executor_runner --model_path ../example_files/vk_add.pte 
I 00:00:00.001837 executorch:executor_runner.cpp:82] Model file ../example_files/vk_add.pte is loaded.
I 00:00:00.001876 executorch:executor_runner.cpp:91] Using method forward
I 00:00:00.001892 executorch:executor_runner.cpp:138] Setting up planned buffer 0, size 48.
I 00:00:00.252948 executorch:executor_runner.cpp:161] Method loaded.
I 00:00:00.253084 executorch:executor_runner.cpp:171] Inputs prepared.
I 00:00:00.254402 executorch:executor_runner.cpp:180] Model executed successfully.
I 00:00:00.254425 executorch:executor_runner.cpp:184] 1 outputs: 
Output 0: tensor(sizes=[1], [2.])

Thanks for your help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@SS-JIA SS-JIA

Labels
module: vulkan triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@SS-JIA @Jack-Khuu @alexggener