lipreading-in-the-wild-experiments

This repository contains my experiments with lip reading using deep learning in Keras. I train and test on the LRW dataset.

process-lrw/

Codes to convert the videos in LRW Dataset to: - frames pertaining to words - mouth area of those frames - audio

Instructions are provided in README file in directory.

shape-predictor/

Directory to place the "shape_predictor_68_face_landmarks.dat" file, required by process-lrw and head-pose

image-retrieval/

Codes and files --- considering the lipreader as an image retrieval system

head-pose/

Codes and files --- to compute head pose in all frames in LRW dataset (extracted using process-lrw)

Head pose is determined using voletiv/deepgaze (my fork of deepgaze).

Instructions are provided in README file in directory.