Simplify vecmem/alpaka interaction

[StewMH]

Move all ifdefs to a single file. Might need some further thought (naming, what we do for SYCL) but makes things better for now already.

[krasznaa]

Remind me Stewart, how does Alpaka handle the platform selection during its build itself? How can you query Alpaka for the current platform that the compilation is happening for?

Because we should integrate better with that system. I would hope that we could create something like a "trait class" that would provide us with the appropriate VecMem types, based on some "Alpaka type".

But I know very little about the implementation of Alpaka. So you'd have to tell me how that could or couldn't work in practice... 🤔

[krasznaa]

I could accept such a header as a "private header". Something under https://github.com/acts-project/traccc/tree/main/device/alpaka/src. But we should not expose any of these macros in a "public header".

[StewMH]

At compile time you can query what accelerator you're using via alpaka::accMatchesTags. So we could do e.g.

if (alpaka::accMatchesTags<TAcc, ::alpaka::TagGpuCudaRt>)
  my_device_memory_resource = vecmem::cuda::device_memory_resource;

but I don't know if this runs into problems with types. Not immediately sure how we avoid ifdef'ed includes either.

EDIT: this now reads read compile-time not runtime.

[krasznaa]

🤔 Again, with very little actual knowledge about Alpaka, it seems to me that the exact type of the "Alpaka queue" would allow you at compile time to decide what type of backend you're running on, no?

In https://github.com/acts-project/traccc/blob/main/device/alpaka/src/utils/utils.hpp the queue type is set up in a pretty simple way at the moment. But from that, I see that there are a bunch of different "accelerator types" that the queue could be created with.

• https://github.com/alpaka-group/alpaka/blob/develop/include/alpaka/acc/AccCpuSerial.hpp
• https://github.com/alpaka-group/alpaka/blob/develop/include/alpaka/acc/AccGpuCudaRt.hpp
• https://github.com/alpaka-group/alpaka/blob/develop/include/alpaka/acc/AccGpuSyclIntel.hpp
• etc.

We should find a way to decide about the correct VecMem types based on which of this template type is used in alpaka::Queue. 🤔

Generally, if we want to use Alpaka seriously, we'll need some type of plugin system in our code. One that could determine at runtime which backends the various algorithms were compiled for. And would be able to instantiate the algorithm compiled for the appropriate backend, based on some job configuration option. (This is sort of what CMS does as I understand it.)

For now it's okay to only compile our Alpaka code for a single backend at a time. But even then, the selection of the VecMem types should be done in "some templated way", so that we would only have to choose the appropriate Alpaka types for the build in a single place, and everything else would be derived from those.

So: Do you have experience with using "traits"? I could point you to some examples of what sort of thing I have in mind.

[StewMH]

Not familiar with traits: please do send me examples of what you have in mind.

[krasznaa]

Forgot about this one... So, a "trait" is something that behaves like:

struct CUDACopy {};
struct SYCLCopy {};

struct CUDAQueue {};
struct SYCLQueue {};

template <typename T>
struct CopySelector {};

template <>
struct CopySelector<CUDAQueue> {
   using copy_type = CUDACopy;
};

template <>
struct CopySelector<SYCLQueue> {
   using copy_type = SYCLCopy;
};

With such a piece of code, you could do something like:

template <typename QUEUE>
void useQueue(QUEUE& q) {
   // Create a copy object that can handle the queue.
   typename CopySelector<QUEUE>::copy_type copy{};
   ...
}

You can find a lot of complicated examples for "traits" for instance in:

• https://github.com/acts-project/vecmem/blob/main/core/include/vecmem/edm/details/view_traits.hpp
• https://github.com/acts-project/vecmem/blob/main/core/include/vecmem/edm/details/schema_traits.hpp
• ...

Yepp, templating can get pretty complicated pretty quickly... 😦

[StewMH]

Thanks Attila. I can look at this. In the meantime can I go ahead with this version after moving it into a private header?

[krasznaa]

Yes, a "private header version" would be okay for the time being. 👍