How to match subsampled-extracted embeddings and original discrete labels

[Jinwoo-Yi]

Hi, I have 19-channel human EEG data with 60,000 timepoints and discrete label sequence of emotional valence with the same timepoints.

Considering high sampling rates of EEG (500Hz here) and complex nature of human emotion, I customized 'offset250-model-25x-subsample' to train CEBRA-behavior model.
Using this model, I obtained (2400, 3) (i.e., (60000/25, num_output)) latent embeddings.

For visualizing embeddings with color-coding valence labels, I need to subsample discrete labels synchronized the timepoint of extracted embeddings. In my guess, it may work to conduct sampling every 25th label but I'm not still sure.

Here's my question: How can I match the timepoint of embeddings (with subsampled) and original discrete labels?
I attach the visual illustration of my question!

p.s.) I'm going to apply CEBRA for my MSc thesis from now on. I guess there will be additional questions, so I will see you often from now on XD

[MMathisLab]

Welcome! Actually in our plotting function it will handle the original input labels, since the model is more about the "receptive field" of your data, it doesn't alter your initial input data, so that should work to post-hoc color the embedding. See the Allen neuropixels demo notebook as there we subsample by 4x as well! Hope that helps

[Jinwoo-Yi]

@MMathisLab Thank you for your swift response. I appreciate your help. I am still grappling with the issue of extracted embedding sizes and their compatibility with valence labels. My goal is to not only visualize the embeddings but also employ them for a classification task, specifically predicting valence labels with extracted embeddings as your 'decoding analysis'.

Here’s where I’m stuck: To conduct the classification task, it's crucial that the length of the embeddings matches that of the valence labels. However, due to the 25x subsampling, the length of the embeddings is currently only 0.25 times the length of the labels (i.e., 2400 = 60,000 / 25).

In the Allen neuropixels demo, I understood that the target of classification (movie frames) had a 4x lower sampling rate than the neuropixel data. Consequently, even after 4x subsampling during training, the latent embeddings of the neuropixel data and movie frame labels ended up with the same length. Unfortunately, in my case, the valence labels and EEG data share the same sampling rates, making it difficult to follow the approach used in the neuropixel demo. This discrepancy is causing me significant confusion.

My question is this: Should I explore alternative encoders that do not rely on subsampling methods? Would these encoders allow me to match the timepoints of the extracted embeddings through the 'transform' function while inputting discrete labels, thus resolving the issue of mismatched lengths?

[MMathisLab]

I personally would not downsample that kind of data - you can make a larger receptive field of the model by using the time offset in the model definition. Others have used EEG data with cebra without sub-sampling, so that seems the better approach. Then you'll avoid having to downsample your labels.

[Jinwoo-Yi]

@MMathisLab Thanks! I'll try this one :D