README.md

Awesome Papers on Automated-Robustness-Testing-for-NLP

Why is testing DNNs important?

DNNs are modern software being deployed everywhere. Like other software these must be tested for corner cases(when the software is likely to be problematic).

Why is testing DNNs hard?

DNNs have too many parameters: too many neurons. Manually finding corner cases is too difficult. Need automated testing , i.e. generating automatically corner cases for large DNNs.

General Intro Position Papers/ Blogs

1. DeepMind Medium Blog
2. General Survey of Testing in ML

Neuron Coverage Based

1. GrayBox Testing: DeepTest
2. White Box Gradient Based Testing
3. DeepCT
4. Concolic Testing for Deep Neural Networks
5. FuzzTesting -- Augmentation
6. Testing Deep Neural Networks- Symbolic Execution
7. MCTS based

Fuzzing Based

1. FuzzTesting
2. TensorFuzz
3. DLFuzz
4. NeuFuzz

Testing for NLP Deep Models

1. checklist
2. errudite
3. Semantically Equivalent Adversarial Rules for Debugging NLP Models
4. Are Red Roses Red?Evaluating Consistency of Question-Answering Models
5. Robustness Verification for Transformers
6. Towards a Robust Deep Neural Network in Texts: A Survey
7. Certified Robustness to Adversarial Word Substitutions