Compute shader compile fails with "Compiler encountered an internal error."

[squidbus]

Re-posting from KhronosGroup/MoltenVK#2363 as this is specific to shader translation.

The below compute shader and pipeline are producing the following error from vkCreateComputePipelines in MoltenVK:

[mvk-error] VK_ERROR_INITIALIZATION_FAILED: Compute pipeline compile failed (Error code 3):
Compiler encountered an internal error.

The crash report for MTLCompilerService says the following:

unable to legalize instruction: %62:_(s32), %63:_(s1) = 145 %59:_, %55:_
Context:
%62:_(s32), %63:_(s1) = 145 %59:_, %55:_
%59:_(s32) = 181 %58:_(s32)
%55:_(s32) = 90 %57:_(p64) :: (dereferenceable load (s32) from `i32 addrspace(64)* bitcast (i8 addrspace(64)* getelementptr inbounds ([28 x i8], [28 x i8] addrspace(64)* @memorycache0, i32 0, i32 8) to i32 addrspace(64)*)`, !tbaa !50, !alias.scope !54, !noalias !61, addrspace 64)
%58:_(s32) = 19 $r10
%57:_(p64) = 193 %7:_, %17:_(s64)
%7:_(p64) = 71 @memorycache0
%17:_(s64) = 120 i64 8
(in function: agc.main)

Compute shader (SPIR-V and generated MSL included): https://gist.github.com/squidbus/50273067f4bd89752473c04683524a93

Pipeline:

• Push Descriptor Layout:
  • Uniform Buffer at Binding 0
  • Uniform Texel Buffer at Binding 1
  • Storage Image at Binding 2
• Push Constants: Offset = 0, Size = 104

System: M3 Max w/ macOS 15.1 Beta 5 (and macOS 15)

This seems to be coming from the last line with the image write, as if I replace the coordinates argument with a fixed zero vector it compiles. Note that there is no error if I use the command line shader compiler, only when compiling a pipeline at runtime.

It looks like this is a Metal compiler bug (which I will try to report upstream - although I'm not sure how much attention feedback reports get), but is this something that could be worked around in SPIR-V to MSL translation?

[HansKristian-Work]

This isn't exactly actionable for me as-is. Replacing coordinates with 0 is surely not a "fix" for anything? Can you propose a concrete compiler workaround?

Generally I'm reluctant to workaround obvious compiler bugs like this unless I'm forced to through MoltenVK conformance or something like that. If the workaround is really simple, it's more palatable.

[squidbus]

I'm not suggesting replacing the coordinates with 0 as a fix, just stating it to note that part of the code is likely the source of the compiler error.

Admittedly I don't have any clue how to go about debugging Apple's internal shader compiler, so I'm not sure what the exact problem is or how to best work around it. I tried tweaking the SPIR-V a bit (generated itself by a recompiler) and wasn't able to significantly change how SPIRV-Cross would output the resulting image write statement one-liner. I also haven't gotten any response back on the FB I filed for this yet.

[HansKristian-Work]

Just noting that I have no way of fixing this on my own, so if a compiler workaround is desired, someone needs to submit PR. Ideally a solution is found first, and I can see how reasonable the workaround is to land in the repo.

[squidbus]

Did some more investigation with the Metal shader and it seems to be related to these unsigned mulhi calls:

_101._m1 = mulhi(cbuf_2.data[_83], _99);
...
_131._m1 = mulhi(cbuf_2.data[_79], _127);

I can replace them with regular multiplication, or cast to signed int, or replace with a different function like min and it will compile.

[HansKristian-Work]

I can replace them with regular multiplication

As in, full 64-bit multiplication instead?

[squidbus]

I can replace them with regular multiplication

As in, full 64-bit multiplication instead?

No, just 32-bit multiplication. Obviously incorrect, but it compiles.

I did try casting to 64-bit, multiplying, and shifting the result, but if the bit shift is exactly 32 then it also fails to compile (shift of 31 or lower, or 33 or higher, does compile).

[HansKristian-Work]

That sounds cursed enough I have no idea how this can be realistically worked around in SPIRV-Cross.

[squidbus]

Actually it turns out the crash had just moved to a different mulhi, if I do this:

[[clang::optnone]] uint spvUMulHi(uint l, uint r)
{
    return ((uint64_t) l * (uint64_t) r) >> 32;
}

And then replace all instances of mulhi with spvUMulHi, it compiles. Do you think that would be acceptable as a workaround? Or since its for OpUMulExtended, something similar that reuses the 64-bit multiply for both the low and high parts.

[squidbus]

Actually even just doing this for Op*MulExtended is working:

template<typename T, typename U>
[[clang::optnone]] T spvMulExtended(U l, U r)
{
    return T{l * r, mulhi(l, r)};
}

So... I'm not really sure why but it seems that as long as its encapsulated like this it compiles fine?