Evolutionary Scale Modeling (ESM) for RNA
Built based on ESM (A Transformer protein language models from Facebook AI Research).
pip install torch
pip install fair-esm
./exp.sh Training script for ESM.
RNA codens as tokens.
A pre-trained model (35M) on snRNA dataset:
| Scale | Coden Size | Number of layers | Embedding dim | Attention heads | Weights |
|---|---|---|---|---|---|
| 35M | 3 | 12 | 480 | 20 | 🤔 |
Downstream tasks:
