Unsupervised-Optical-Flow

Pre-requisite for rigid flow

We used the training pipeline from monodepth2

Using the pose estimator module, we can compute the displacement at a pixel level from source to target. We extracted a rigid flow field, which is valid only for the static objects in the scene and captures the car's motion relative to the environment but fails to capture objects in motion.

mkdir models

A pretrained model for the UPB dataset is available here. Copy all the files into the "models" directory.

Create dataset

mkdir raw_dataset

Copy the video recodings in the "raw_dataset" directory. A sample of the UPB dataset is available here.

Split the videos in train and test/validation:

python3 scripts/split_dataset.py \
 --src_dir raw_dataset
 --dst_dir split_scenes

Transform videos into frames for training:

python3 scripts/create_dataset.py \
 --src_dir raw_dataset\
 --dst_dir dataset \
 --split_dir split_scenes \
 --dataset upb

Train model

python3 train.py \
 --dataset upb \
 --batch_size 12 \
 --num_epochs 20 \
 --scheduler_step_size 15 \
 --height 256 \
 --width 512 \
 --model default

Pre-trained model

mkdir -p snapshots/checkpoints/

Download pretrained model from here into the "snapshots/checkpoints" directory

Test model

The checkpoint should be present in "snaphsots/checkpoints" directory

python3 test.py \
 --video_path raw_dataset/fc0768f4d5a34d12.mov \
 --save_gif teaser/fc0768f4d5a34d12.gif \
 --model_name default.pth