A collection of papers and resources related to security issuses of Pre-trained Models.
The organization of papers refers to our survey "New Emerged Security and Privacy of Pre-trained Model: a Survey and Outlook".
Please let us know if you find out a mistake or have any suggestions by email: [email protected]
If you find our survey useful for your research, please cite the following paper:
@article{yang2024new,
title={New Emerged Security and Privacy of Pre-trained Model: a Survey and Outlook},
author={Yang, Meng and Zhu, Tianqing and Liu, Chi and Zhou, WanLei and Yu, Shui and Yu, Philip S},
journal={arXiv preprint arXiv:2411.07691},
year={2024}
}
| Model | Available | Modal | Size(B) | Base Model | Release Time |
|---|---|---|---|---|---|
| GPT-1 | Open-source | text | - | Transformer (decoder) | Jun-2018 |
| BERT | Open-source | text | 330MB | Transformer (Encoder) | Oct-2018 |
| CLIP | Open-source | multi | 400MB | Resnet / Transformer (Encoder) | Jan-2021 |
| Model | Available | Modal | Size(B) | Base Model | Release Time |
|---|---|---|---|---|---|
| GPT-3 | Close-source | text | 6B/175B | GPT-2 | May-2020 |
| GPT-3.5 | Close-source | text | - | GPT-3 | Mar-2022 |
| Flamingo | Close-source | multi | 3B/9B/80B | CLIP+Transformer (decoder) | Apr-2022 |
The target model comprises two critical components: (1) the input message and (2) the model structure along with its parameters. Attacks can lead to changes in either or both components of the target model. Hence, this article categorizes the impacts on these model components by analyzing the changes observed before and after an attack.
