C3D_BICLSTM

Prerequisites

1. Tensorflow-0.11
   
2. Tensorlayer (commit ba30379f1b86f930d6e86e1c8db49cbd2d9aa314)
   git clone https://github.com/zsdonghao/tensorlayer.git
   git checkout ba30379f1b86f930d6e86e1c8db49cbd2d9aa314
   

The original Tensorlayer does not support the convolutional LSTM, so the tensorlayer/layers.py needs to be replaced with tensorlayer-layers.py.

Get the pretrained models

The pretrained models used in training processes and our trained models can be obtained in the link: https://pan.baidu.com/s/1slc2DMd Password: sty6.

How to use the code

Prepare the data

1. Convert each video files into images using extract_frames.sh in the dataset_splits/video2image.tar.gz. Before running extract_frames.sh, you should change the ROOTDIR in extract_frames.sh, so that IsoGD_phase_1 and IsoGD_phase_2 do exist under $ROOTDIR.
2. Replace the path "/ssd/dataset" in the files under "dataset_splits" with the path "$ROOTDIR"
3. run check_files.py to make sure all necessary image files do exist

Training Stage

1. Use training_*.py to finetune the networks for different modalities. Please change os.environ['CUDA_VISIBLE_DEVICES'] according to your workstation.
   

Validating and testing Stage

1. Use testing_isogr_valid.py and testing_isogr_test.py to evaluate the trained models on the validating and testing sets of IsoGD

Citation

Please cite the following paper if you feel this repository useful.
http://ieeexplore.ieee.org/abstract/document/7880648/ http://openaccess.thecvf.com/content_ICCV_2017_workshops/w44/html/Zhang_Learning_Spatiotemporal_Features_ICCV_2017_paper.html

@article{ZhuICCV2017,
  title={Learning Spatiotemporal Features using 3DCNN and Convolutional LSTM for Gesture Recognition},
  author={Liang Zhang and Guangming Zhu and Peiyi Shen and Juan Song and Syed Afaq Shah and Mohammed Bennamoun},
  journal={ICCV},
  year={2017}
}
@article{Zhu2017MultimodalGR,
  title={Multimodal Gesture Recognition Using 3-D Convolution and Convolutional LSTM},
  author={Guangming Zhu and Liang Zhang and Peiyi Shen and Juan Song},
  journal={IEEE Access},
  year={2017},
  volume={5},
  pages={4517-4524}
}

Contact

For any question, please contact

  [email protected]