[SYCL][CUDA] CXX Standard Library Extension for NVPTX backend

[hdelan]

Proposal for CXX standard library support for CUDA backend.

CXX standard library support for SPIR-V backends is provided by IR bundled in .o files such as libsycl-fallback-cmath.o. The IR for each SPIR-V backend is unbundled and linked at compile time, before being further compiled to SPIR-V or device code along with the rest of the source code.

We propose that the NVPTX target should enable CXX standard library functions in the same way. This involves:

1. Compiling the CXX functions to LLVM IR specific to the NVPTX targets and bundling them into the same .o files (such as libsycl-fallback-cmath.o) at the building stage of DPC++.
2. Changing the clang driver so that for the NVPTX backend, the same .o files are unbundled and linked with source code.

There is also the small matter of stripping the cuda-gpu-arch information from the LLVM IR when the .o files are being created; we think we have an approach for this but will only know for certain later down the line.

Please let me know if anyone has any suggestions, and if you think this is a suitable way of implementing CXX stdlib support for the CUDA backend.

[bader]

Following SPIR-V backend design for CUDA/HIP makes sense to me. 👍

CXX standard library support for SPIR-V backends is provided by IR bundled in .o files such as libsycl-fallback-cmath.o. The IR for each SPIR-V backend is unbundled and linked at compile time, before being further compiled to SPIR-V or device code along with the rest of the source code.

Just a minor clarification: if low-level runtime supports corresponding "native device lib" extension then DPC++ compiler will skip this step. So there is an option to provide a math library optimized by the HW vendor.

[jinge90]

The proposal looks good to me. And I remember the support for NVPTX backend is similar to AOT compilation, if so, we can ignore the "native device lib" extension since it only works for jit compilation currently.

Thanks very much.

[pvchupin]

@jinge90, could you please review?

[hdelan]

The first stage of implementation is roughly working, however there are some issues around using common functions like cos, sin, exp etc. These are recognized by clang as things that can be replaced by LLVM instructions later on, so the funcs are replaced before the libdevice functions are linked. This causes errors later on when say llvm.cos.f32 resolves to the instruction fcos which is invalid for NVPTX. The approach that we think will explore first is to modify CGBuiltin.cpp so that instead of the funcs resolving to the LLVM builtins, to resolve instead to the libspirv func, (which will resolve to __nv funcs for NVPTX backend). This would mean specializing CGBuiltin slightly for the SYCL NVPTX triple.

Another option is to change the fcos instruction inserted later in LLVM, however this kind of specialization in LLVM is not seen often in the codebase.
Again another option is to force include a special header that will redirect say std::cos to the correct builtin, however this is a bit messy and inelegant.

Let us know your thoughts, or if you think there is another approach we can take. We think the first option is the best approach as it stands.

@Naghasan @AerialMantis

[bader]

Will something like -fno-builtin-<arg> or -fno-builtin help here?

@jdoerfert, FYI.

[Naghasan]

We considered that option, but it may degrade performances (if not fail to compile).

[jdoerfert]

We have a libm.a with embedded code for amdhsa and nvptx. The problem is where to uptream it.
We'll probably put it in a "GPUlibs" folder for now. You link it in during the device LTO stage and you get fcos -> __nv_fcos translation. It's build through the CUDA/HIP math headers we pre-include right now. See the post for some more plans:
https://discourse.llvm.org/t/nvptx-codegen-for-llvm-sin-and-friends/58170/11?u=jdoerfert

[Naghasan]

(1) if -fno-errno is set (or equivalent) which we don't set for CUDA/HIP/OpenMP (just yet)

Thanks, that's the difference. SYCL when targeting nvptx64 actually sets no errno (but does when targeting SPIR)...

[bader]

@bader, Why would we want the devicelib indirection? I can see that for non C/C++ standard functions but especially for math, cosf is way better as a target independent intermediate format, no?

I don't say that we should upstream the workflow I described as is (IMHO we can drop at least one library layer), but I think there a few important requirements to make GPUlibs useful for multiple languages and targets.

1. "Language neutral API". I tend to agree that cosf can be a target independent intermediate format, but I'm not sure if it's okay to use for different programming languages clang supports. For instance, cos (double), cosf (float) standard C library names potentially might conflict with user defined functions OpenCL C. OpenCL C has it's own set of math built-ins instead of standard C math library, which are overloadable cos (double), cos (float) and cos (half). One of the solutions here is using symbols in the compiler reserved namespace e.g. GPUlibs expose __cos (double), __cos (float) APIs which we can use to implement both C standard math function and OpenCL C math built-ins. I think this namespace rule applies to all C/C++ based languages, but it would be great to check if it also works for Fortran and other languages targeting GPUs.
2. GPUlibs should support runtime dispatch to target specific implementation. It's important for targets programmed though virtual targets like SPIR-V. I think we already have everything in place to support for versions of dispatching i.e. at compile time (with target specific macro) and at runtime (with linking target specific bitcode library).
3. GPUlibs should allow configuring precision of math functions implementation. OpenCL/SYCL built-in math functions has specific accuracy requirements, so their implementation might be different from C/C++ standard math library implementation. In addition to that users might expect different implementation to be used when using different compiler knobs configuring floating point computation. @andykaylor is currently working on solution to enable such configuration for FP functions on GPU, including intermediate format representation for math functions.

[hdelan]

The current behaviour of Cuda ToolChain is:

bool IsMathErrnoDefault() const override { return false; }

Which as far as I can tell is equivalent to fno-math-errno being on by default. Is this subject to change once the upstream approach is complete?

Changing this so that it returns true for Cuda ToolChain would allow us to use libdevice by the same exploit as the SYCL ToolChain is using. Let us know if there should be a different approach, perhaps for both SYCL and Cuda toolchain

[jdoerfert]

I get what you are saying. For 1., I'd argue it is OpenCL that needs to use different names or map to C names. For 2., cosf and friends should be fine. For 3., I agree that requirements not matching cosf should result in other calls.

Upstream will soon start with some form of "GPULibs", would be great if we work on that together.

[bader]

Upstream will soon start with some form of "GPULibs", would be great if we work on that together.

Sure. I'm interested to have GPU side libraries in llorg, which we can re-use for SYCL. 👍

[hdelan]

#6482

[keryell]

@Ralender @lforg37 any feedback?