Kronecker product of two ggml_tensor: How to?

[stduhpf]

I'm currently trying to implement Lycoris LoKr support for stable-diffusion.cpp. This uses a Kroenecker product to achieve a higher rank matrix with fewer parameters than regular low rank decomposition.

$$A \otimes B = \begin{pmatrix} a_{11}B & a_{12}B & \cdots & a_{1n}B \\\ a_{21}B & a_{22}B & \cdots & a_{2n}B \\\ \vdots & \vdots & \ddots & \vdots \\\ a_{m1}B & a_{m2}B & \cdots & a_{mn}B \end{pmatrix}$$

(easily generalisable to arbitrary dims)

The problem is, while Kronecker product is really to understand conceptually, and trivial to implement in regular code, I have no idea how to implement it properly with ggml. I just need it to work for 2D matrices.

At first I thought ggml_out_prod() with some reshaping after would be able to achieve it, but I think I was mistaken about the function of ggml_out_prod().

[ggerganov]

Maybe ggml_mul(ctx, a, ggml_repeat(ctx, b, a))? Not 100% sure though.

[stduhpf]

No, this is not qute it, ggml_repeat(ctx, b, a) output is the same size as a, but it should be the size of a multiplied by the size of b

[stduhpf]

But that lead me on the right path. I believe ggml_mul(ctx, ggml_upscale_ext(ctx, a, a->ne[0] * b->ne[0], a->ne[1] * b->ne[1], a->ne[2] * b->ne[2], a->ne[3] * b->ne[3]), b); should work. At least it seems to be the correct shape.

Edit: it crashes (because upscale only works with f32 apparently, otherwise it works)