Facial Beauty Predictor

A deep learning model based on FaceNet and MTCNN to predict a beauty score for faces in images. The CNN outperformes the state-of-the art by up to 18% (2019).

Included are scripts for generating features from images, training regressors, as well as a async server for inference based on gunicorn / gevent.

Based on

tindetheus

FaceNet

MTCNN

Requirements:

• python 3.7
• pipenv

Installation

pipenv install --dev

Quick Start

• Download SCUT dataset
• Download HotOrNot dataset
• Download the FaceNet model 20170512-110547 and extract it into the data directory
• Convert datasets with:
  • python scripts/convert_scut.py --db-dir <path/to/db/dir>
  • python scripts/convert_tinder.py --db-dir <path/to/db/dir>
  • python scripts/convert_hotornot.py --db-dir <path/to/db/dir>
• Generate features for SCUT dataset once and store them to the disk:
  • python scripts/generate_features_async.py --db data/scut.pkl --output-dir data/scut
• Train regressor models with
  • python scripts/train_regressor.py --db data/scut.pkl --output-dir data/scut/mtcnn-facenet --features data/scut/mtcnn-facenet/features.npy
• Compare regressors:
  • python scripts/compare_models.py --db data/scut.pkl --output-dir data/scut/mtcnn-facenet --models-dir data/scut/mtcnn-facenet/models --features data/scut/mtcnn-facenet/features.npy
• Generate model trained on all the dataset:
  • python scripts/train_regressor.py --db data/scut.pkl --output-dir data/scut/mtcnn-facenet --features data/scut/mtcnn-facenet/features.npy --no-split
• Generate features for Tinder dataset once and store them to the disk:
  • python scripts/generate_features_async.py --db data/tinder.pkl --output-dir data/tinder
• Infer results on tinder dataset:
  • python scripts/infer.py --db data/tinder.pkl --features data/tinder/mtcnn-facenet/features.npy --model data/scut/mtcnn-facenet/models/all/sklearn.linear_model.base.LinearRegression_1.pkl

Those steps can be repeated for a mtcnn-only backbone (put --backbone mtcnn flag where necessary and replace mtcnn-facenet with mtcnn)

Results

SCUT Dataset

FaceNet features

Regressor	PC
Lasso	0.846
Ridge	0.872
Linear	0.872

FaceNet + MTCNN features:

Regressor	PC

@TODO (note: was slightly better than Facenet features only)

MTCNN only features

Regressor	PC

@TODO

HotOrNot Dataset

FaceNet features

Regressor	PC
Linear	0.536
Lasso	0.550
Ridge	0.567

FaceNet + MTCNN features

Regressor	PC

@TODO

MTCNN only features

Regressor	PC

@TODO