How to do the automatic safety evaluations

[tivon-x]

Hello!

This is very nice work!

I kindly want to know how to do the automatic safety evaluations. According to the paper, you use a safety critique llm for the evaluations. Will you release the safety critique llm in the future? Or are there any other methods for the automatic safety evaluations?

[IS2Lab]

Thanks for your recognition of our work. At present, we have no plans to release our safety critique llm in the short term. However, we are happy to provide support to help you with safety evaluations. If you require automatic safety evaluations, please provide a jsonl file including traceid, prompt, and the corresponding response and send it to [email protected] (if you have any questions about the required submission files, please contact this email first.) We will evaluate and return the results according to the order of requests.

In addition, for other automatic safety evaluations, we introduced and analyzed various methods in our paper. You can refer to the paper.

Thanks again for your understanding and support!