Skip to content

Latest commit

 

History

History
21 lines (18 loc) · 952 Bytes

README.md

File metadata and controls

21 lines (18 loc) · 952 Bytes
Raw

Automated-HCI

Use CNN's to predict user behavior on mobile UI's.

We assume user interactions are broken down into 7 classes:

  1. Do nothing
  2. Tap top left quadrant
  3. Tap top right quadrant
  4. Tap bottom left quadrant
  5. Tap bottom right quadrant
  6. Swipe left/right
  7. Swipe up/down

Our training set consists of 10000 UI's for each class. Our testing set consists of 500 UI's for each class.

Usage:

  1. Download the RICO interaction traces dataset (http://interactionmining.org/rico) and put it in this directory.
  2. Run 'python3 generate_labels.py' to generate the screenshot-to-gesture labels and preprocess images. This can take a while.
  3. Run 'python3 train.py', adjusting the step size as the model converges. We recommend starting with a step size of 0.001.
  4. Run 'python3 test.py'.
  5. To visualize ROC curves, run 'python3 plotting/plot_ROC_curves.py'.
  6. To visualize the confusion matrix, run 'python3 plotting/plot_confusion_matrix.py'