Separate grid search for contrastive vs decoding loss?

[asafbenj]

Hi,
Does it make sense to optimize hyper-params for the contrastive loss separately from decoding? I.e. is the embedding of the model with the lowest contrastive loss expected to be the best input for decoding, or should we optimize them together, even if it's highly inefficient (e.g. lots of unlabeled data and a bit of labeled data)?
Thanks!

[MMathisLab]

We optimize them separately; I would not go on to decoding until one has a consistent encoding model (i..e running the same parameters many times gives you the same latents).

[asafbenj]

Thanks for the fast response!
Just to make sure - you mean running the same params and same data, just different seed many times, or same params and different subsets of the data? And re' "same latents" - I'll re-examine the details of the paper and demos, but generally speaking, you mean qualitatively similar structure, or a hard threshold for some measure of similarity?
Thanks again!

[MMathisLab]

You should use the consistency metric