CUDA compile falure, error: user-defined literal operator not found, type name not allowed, and undefines.

[joshuachris2001]

https://asciinema.org/a/jbeHb3zISfeDUTDOyKKq6HXnT

numinous errors compiling ggml-cuda.cu

/usr/include/c++/14.1.1/x86_64-pc-linux-gnu/bits/c++config.h(827): error: user-defined literal operator not found
    typedef __decltype(0.0bf16) __bfloat16_t;
                       ^

/usr/include/c++/14.1.1/type_traits(529): error: type name is not allowed
      : public __bool_constant<__is_array(_Tp)>
                                          ^

/usr/include/c++/14.1.1/type_traits(529): error: identifier "__is_array" is undefined
      : public __bool_constant<__is_array(_Tp)>
                               ^

/usr/include/c++/14.1.1/type_traits(581): error: type name is not allowed
      : public __bool_constant<__is_member_object_pointer(_Tp)>
                                                          ^

/usr/include/c++/14.1.1/type_traits(581): error: identifier "__is_member_object_pointer" is undefined
      : public __bool_constant<__is_member_object_pointer(_Tp)>
                               ^

/usr/include/c++/14.1.1/type_traits(603): error: type name is not allowed
      : public __bool_constant<__is_member_function_pointer(_Tp)>
                                                            ^

/usr/include/c++/14.1.1/type_traits(603): error: identifier "__is_member_function_pointer" is undefined
      : public __bool_constant<__is_member_function_pointer(_Tp)>
                               ^

/usr/include/c++/14.1.1/type_traits(695): error: type name is not allowed
      : public __bool_constant<__is_reference(_Tp)>
                                              ^

/usr/include/c++/14.1.1/type_traits(695): error: identifier "__is_reference" is undefined
      : public __bool_constant<__is_reference(_Tp)>
                               ^

/usr/include/c++/14.1.1/type_traits(731): error: type name is not allowed
      : public __bool_constant<__is_object(_Tp)>
                                           ^

/usr/include/c++/14.1.1/type_traits(731): error: identifier "__is_object" is undefined
      : public __bool_constant<__is_object(_Tp)>
                               ^

/usr/include/c++/14.1.1/type_traits(760): error: type name is not allowed
      : public __bool_constant<__is_member_pointer(_Tp)>
                                                   ^

/usr/include/c++/14.1.1/type_traits(760): error: identifier "__is_member_pointer" is undefined
      : public __bool_constant<__is_member_pointer(_Tp)>
                               ^

/usr/include/c++/14.1.1/bits/utility.h(237): error: __type_pack_element is not a template
      { using type = __type_pack_element<_Np, _Types...>; };
                     ^

/usr/include/c++/14.1.1/type_traits(138): error: class "std::enable_if<<error-constant>, void>" has no member "type"
      using __enable_if_t = typename enable_if<_Cond, _Tp>::type;
                                                            ^
          detected during:
            instantiation of type "std::__enable_if_t<<error-constant>, void>" at line 176
            instantiation of "std::__detail::__or_fn" based on template arguments <std::is_reference<std::allocator<char>>, std::is_function<std::allocator<char>>, std::is_void<std::allocator<char>>, std::__is_array_unknown_bounds<std::allocator<char>>> at line 194
            instantiation of class "std::__or_<_Bn...> [with _Bn=<std::is_reference<std::allocator<char>>, std::is_function<std::allocator<char>>, std::is_void<std::allocator<char>>, std::__is_array_unknown_bounds<std::allocator<char>>>]" at line 1195
            instantiation of class "std::is_nothrow_default_constructible<_Tp> [with _Tp=std::allocator<char>]" at line 528 of /usr/include/c++/14.1.1/bits/basic_string.h
            instantiation of "std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string() [with _CharT=char, _Traits=std::char_traits<char>, _Alloc=std::allocator<char>]" at line 4248 of /usr/include/c++/14.1.1/bits/basic_string.h

15 errors detected in the compilation of "ggml-cuda.cu".
make: *** [Makefile:318: ggml-cuda.o] Error 2

[Nolicx]

Same here

/usr/include/c++/14.1.1/x86_64-pc-linux-gnu/bits/c++config.h(827): error: user-defined literal operator not found
    typedef __decltype(0.0bf16) __bfloat16_t;
                       ^

/usr/include/c++/14.1.1/type_traits(529): error: type name is not allowed
      : public __bool_constant<__is_array(_Tp)>
                                          ^

/usr/include/c++/14.1.1/type_traits(529): error: identifier "__is_array" is undefined
      : public __bool_constant<__is_array(_Tp)>
                               ^

/usr/include/c++/14.1.1/type_traits(581): error: type name is not allowed
      : public __bool_constant<__is_member_object_pointer(_Tp)>
                                                          ^

/usr/include/c++/14.1.1/type_traits(581): error: identifier "__is_member_object_pointer" is undefined
      : public __bool_constant<__is_member_object_pointer(_Tp)>
                               ^

/usr/include/c++/14.1.1/type_traits(603): error: type name is not allowed
      : public __bool_constant<__is_member_function_pointer(_Tp)>
                                                            ^

/usr/include/c++/14.1.1/type_traits(603): error: identifier "__is_member_function_pointer" is undefined
      : public __bool_constant<__is_member_function_pointer(_Tp)>
                               ^

/usr/include/c++/14.1.1/type_traits(695): error: type name is not allowed
      : public __bool_constant<__is_reference(_Tp)>
                                              ^

/usr/include/c++/14.1.1/type_traits(695): error: identifier "__is_reference" is undefined
      : public __bool_constant<__is_reference(_Tp)>
                               ^

/usr/include/c++/14.1.1/type_traits(731): error: type name is not allowed
      : public __bool_constant<__is_object(_Tp)>
                                           ^

/usr/include/c++/14.1.1/type_traits(731): error: identifier "__is_object" is undefined
      : public __bool_constant<__is_object(_Tp)>
                               ^

/usr/include/c++/14.1.1/type_traits(760): error: type name is not allowed
      : public __bool_constant<__is_member_pointer(_Tp)>
                                                   ^

/usr/include/c++/14.1.1/type_traits(760): error: identifier "__is_member_pointer" is undefined
      : public __bool_constant<__is_member_pointer(_Tp)>
                               ^

/usr/include/c++/14.1.1/bits/utility.h(237): error: __type_pack_element is not a template
      { using type = __type_pack_element<_Np, _Types...>; };
                     ^

/usr/include/c++/14.1.1/type_traits(138): error: class "std::enable_if<<error-constant>, void>" has no member "type"
      using __enable_if_t = typename enable_if<_Cond, _Tp>::type;
                                                            ^
          detected during:
            instantiation of type "std::__enable_if_t<<error-constant>, void>" at line 176
            instantiation of "std::__detail::__or_fn" based on template arguments <std::is_reference<std::allocator<char>>, std::is_function<std::allocator<char>>, std::is_void<std::allocator<char>>, std::__is_array_unknown_bounds<std::allocator<char>>> at line 194
            instantiation of class "std::__or_<_Bn...> [with _Bn=<std::is_reference<std::allocator<char>>, std::is_function<std::allocator<char>>, std::is_void<std::allocator<char>>, std::__is_array_unknown_bounds<std::allocator<char>>>]" at line 1195
            instantiation of class "std::is_nothrow_default_constructible<_Tp> [with _Tp=std::allocator<char>]" at line 528 of /usr/include/c++/14.1.1/bits/basic_string.h
            instantiation of "std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string() [with _CharT=char, _Traits=std::char_traits<char>, _Alloc=std::allocator<char>]" at line 4248 of /usr/include/c++/14.1.1/bits/basic_string.h

15 errors detected in the compilation of "ggml-cuda.cu".
make: *** [Makefile:318: ggml-cuda.o] Error 2

EDIT: Found this
https://forums.developer.nvidia.com/t/cuda-12-4-nvcc-and-gcc-14-1-incompatibility/293295
I'm using Arch with

╰─❯ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-linux-gnu/14.1.1/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /build/gcc/src/gcc/configure --enable-languages=ada,c,c++,d,fortran,go,lto,m2,objc,obj-c++,rust --enable-bootstrap --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://gitlab.archlinux.org/archlinux/packaging/packages/gcc/-/issues --with-build-config=bootstrap-lto --with-linker-hash-style=gnu --with-system-zlib --enable-__cxa_atexit --enable-cet=auto --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-default-ssp --enable-gnu-indirect-function --enable-gnu-unique-object --enable-libstdcxx-backtrace --enable-link-serialization=1 --enable-linker-build-id --enable-lto --enable-multilib --enable-plugin --enable-shared --enable-threads=posix --disable-libssp --disable-libstdcxx-pch --disable-werror
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 14.1.1 20240522 (GCC)

and

╰─❯ /opt/cuda/bin/nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Jun__6_02:18:23_PDT_2024
Cuda compilation tools, release 12.5, V12.5.82
Build cuda_12.5.r12.5/compiler.34385749_0

[heartofrain]

Hi, @Nolicx @joshuachris2001 . When I compiled the NVIDIA's CUDA sample vectorAdd in Arch Linux, I came across same errors. My gcc version is 14.1.1 20240720, and my nvcc version is 12.5.r12.5/compiler.34385749_0.
According to this post, the reason for these errors seems to be that the version of gcc does not match the version of nvcc.
NVIDIA's official installation guide lists a table of supported compilers. It shows that only 6.x - 13.2 versions of gcc are supported. (up to today, 2024.7.25)
So, check if the gcc13 is installed:

$ /usr/bin/gcc-13 --version

or install the gcc13 package, or install the lower version gcc12 package from AUR. Or install the appropriate package if you are using other Linux distributions.
Then use the -ccbin argument of nvcc to specify the version of gcc, like this:

/opt/cuda/bin/nvcc -ccbin /usr/bin/gcc-13 #...ellipsis...#

or use HOST_COMPILER as a make argument if possible, like this:

make HOST_COMPILER=/usr/bin/gcc-13

After that I have successfully compiled CUDA sample vectorAdd.

Note:

1. Although gcc13.3 is not listed in NVIDIA's official installation guide, when I compiled vectorAdd with gcc13.3, no error occurred.
2. I have not compiled the whisper.cpp project yet, you can have a try. GOOD LUCK!

[Tiger1218]

I met the same issue and solve it with @heartofrain 's solution, thank you!

[calgray]

I experienced this exact same issue on ArchLinux after updating. Interestingly I have a backup from earlier this year that worked fine with the following slightly older cuda revision:

>>> nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Wed_Apr_17_19:19:55_PDT_2024
Cuda compilation tools, release 12.5, V12.5.40
Build cuda_12.5.r12.5/compiler.34177558_0

and:

>>> gcc --version
gcc (GCC) 14.1.1 20240522
Copyright (C) 2024 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.