Skip to content

Adversarial Attacks on Neural Automatic Essay Scoring Systems; Lee, Michael & Nickerson, Micah

Notifications You must be signed in to change notification settings

mjnickerson/csci-89a-final_project

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

39 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Adversarial Attacks on Neural Automated Essay Scoring Systems

Michael Lee & Micah Nickerson

Harvard University Summer

CSCI-S-89A- FINAL PROJECT

We evaluate the success of NLP Adversarial Techniques against an Automated Essay Scoring System "Black Box."

Part 1 - Creating a Dual Scoring CNN system, as a Black Box to test.

  • See:
    • AES Blackbox- Dual CNN.ipynb

Part 2 - Generating Adversarial Attacks.

  • See:
    • Adversarial Attack Perturbation Creator.ipynb
    • Anchor Model using AES CNN Model.ipynb

Part 3 - Applying the attacks, and evaluating the results.

  • See last section of:
    • AES Blackbox- Dual CNN.ipynb

Prerequisites

  • All libraries required are listed in built with below.

  • GloVe Embedding Vector sets (by Stanford University) must be downloaded into Data set/GloVe

Installing

  • Final Blackbox H5 models are available upon request

  • Recommended to run code on Google colab

  • Notebook "AES Blackbox..." is designed to both train the black box, evaluate the black box, and adversarially attack the black box. The input line specifying this function is in FIRST code section of notebook "AES Blackbox...".

Built With

Contributing

Please contact us for details on our code of conduct, and the process for submitting pull requests to us.

Versioning

Versioning has not been established - project is to be published once.

Authors

License

This project is licensed under the MIT License.

Acknowledgments

  • Thank you to Professor Zoran Djordjevic and the TA staff of Harvard University Summer CSCI-S89A for their input and support.

  • This project is based on a foundational paper by David Lang and Klint Kanopka at Stanford University.

    • Lang, K. K. a., Adversarial Examples for Neural Automatic Essay Scoring Systems. Stanford University cs224n/15720509, (2019).
  • Also thanks to Yoon Kim for his work on classifying sentences with CNN, which was a primary reference.

    • Kim, Yoon. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882, (2014).

Releases

No releases published

Packages

No packages published