Question: How to use templated classes with user defined SIMD functions and non-SIMD functions.

[OliverWangData]

Hey all, apologies this is more of a question about how the library works rather than a specific bug. I've read through the documentation quite a few times but I still don't understand how to use the library in my use case.

Let's say I have some templated user defined functions that act on vectors. The setup is similar to skeleton-inl.h:

#if defined(FIXEDPOINT_SIMD_H_) == defined(HWY_TARGET_TOGGLE)
#ifdef FIXEDPOINT_SIMD_H_
#undef FIXEDPOINT_SIMD_H_
#else
#define FIXEDPOINT_SIMD_H_
#endif

#include "hwy/highway.h"

namespace FixedPoint::HWY_NAMESPACE
{
    template <size_t B>
    using VecType = Vec<ScalatableTag<TypeSelector<B>::Type>>;

    template <size_t B>
    inline constexpr VecType<B> CustomOp(VecType<B> a, VecType<B> b) {
        return hwy::HWY_NAMESPACE::Add(a, b);
    };
}

#endif

I also have a templated class that has both regular functions, and functions that use my user-defined fixed point operations. And I want to use runtime dispatch (Note: hwy boilerplate removed below):

template <size_t B>
class Algorithm
{
public:
   void Clear() {
       // Non simd code
    };

protected:
    using FixedPoint::HWY_NAMESPACE::VecType;

    VecType<B> Process(VecType<B> v1, VecType<B> v2) {
        // Only uses functions in the FixedPoint::HWY_NAMESPACE
    }
}

My goal is to be able to use this class inside code that does not does not need to follow the google highway BEFORE/AFTER_NAMESPACE and HWY_NAMESPACE conventions.

I have a few questions regarding this setup:

Would this Algorithm class need to exist inside some [nested]::HWY_NAMESPACE?
In the documentation, it says that user-defined functions reside in that namespace. So I assume since the class contains a user-defined function, it should also exist in that namespace. Is that correct?

Would the Process(v1, v2) function need to be prefixed with HWY_ATTR?
Reading the documentation, it says that functions using SIMD either need to reside inside that before/after macro, or have HWY_ATTR. Does this mean strictly the functions provided by hwy::HWY_NAMESPACE? Or anything to do with SIMD and anything simd-derived? Does it need to be prefixed because it's using VecType in the signature? Or is that alright, but it needs to be prefixed because it's using user-defined fixedpoint "-inl.h" operations? Or does it not need to be prefixed at all?

I've tried consulting the examples, but I don't see an example with a situation where a class is templated (requiring functions to be defined in the header). I would prefer the class itself to be usable outside of the highway context (Use case: Unreal Engine UCLASS using this algorithm class), but does this algorithm class need a "non-simd" wrapper class for this?

[jan-wassenberg]

Hi, thanks for reaching out. I am curious if you've already got this working?

Some quick comments:

1. Using HWY_NAMESPACE is required for runtime dispatch mode, because it compiles your code several times. HWY_NAMESPACE evaluates to a different namespace each time, which is required in order to prevent redefinition.

2. HWY_ATTR is required for any function that uses SIMD intrinsics or Highway ops, because that is how we tell the compiler which targets to enable via pragma. Some compilers require that in order to allow the intrinsics to be used, though not MSVC.

For an example of exporting a function template, check out the usage of HWY_EXPORT_AND_DYNAMIC_DISPATCH_T in https://github.com/google/gemma.cpp/blob/dev/gemma/gemma-inl.h.

Hope this helps?

[OliverWangData]

Hey Jan, thanks for the reply and the link!

I think I've got something that could potentially work, but I'm slowly resolving other issues that are cropping up. But I'm no longer stuck with the issue above. The setup I'm currently theorizing and trying to get working is to define a non-simd base class with the non-simd functions, and have a simd derived class defined inside the [nested]::HWY_NAMESPACE with simd-exclusive functions. That way non-simd code can box the object into the base class, whereas simd code can use the derived class.

I'd like to test this out more thoroughly and see if any compiler / cpu architecture edge case issues crop up, but I'm still running into various small issues trying to get this working. Hopefully it does but we'll see!

[jan-wassenberg]

You're welcome!
The base class approach sounds good. You may find #2364 interesting, it discusses that with some example code.