Question regarding perplexity values #6
Description
In the paper associated with RDesign, perplexity values are not reported and claimed to be an unsuitable metric for RNA design (Appendix E.3).
However, in the code, we can see that perplexity is computed and the training logs suggest very unusual perplexity values for the released model: https://github.com/A4Bio/RDesign/blob/master/checkpoints/log.log#L626
Final perplexity:
- Train: 1.0627
- Validation: 22.1759 (suggests overfitting?)
- Test: 3.6372 (different from what's reported in the paper?)
What is the reason that the model produces perplexity values at very different ranges here?