Why are CEBRA embeddings in 3D usually lying on a circle?

[timonmerk]

Thanks again for providing this great tool and also keeping the discussion here on GitHub!

I wondered, based on the results presented in the paper, but also based on our own results, why CEBRA embeddings are usually lying on a sphere?

This is purely out of curiosity.. Is it related to the loss InfoNCE loss function? In the beginning I thought that it was related to the "circular" task presented in Figure 1 where the rat is running on a track forth and back. But for the other embeddings that also seems to be the case.

[stes]

@timonmerk , this a choice you can influence by picking loss function and model properties.

The offset10-model and cosine similarity metrics will learn an embedding on the hypersphere, while the offset10-model-mse with euclidean similarity gives you an embedding in Euclidean space (cf eg the behavior embedding on paper Figure 2 which is trained with this loss).

It depends a bit which latent space is more suitable for your application, but you are free to use either In CEBRA.

[MMathisLab]

And note that it need not be in a ring; figure 3 shows embeddings on the sphere that are not in a ring (district 8 directions), and on animal behavior in our new work amadeusGPT there is a cebra example that shows continuous motion on a "ring" plus discrete states, so it really depends on the input data too.