Motion Blur Decomposition with Cross-shutter Guidance

Arxiv | Paper | Supp | Project page

Xiang Ji, Haiyang Jiang, Yinqiang Zheng

The University of Tokyo

This repository provides the official PyTorch implementation of the paper.

TL;DR

Inspired by the complementary exposure characteristics of a global shutter (GS) camera and a rolling shutter (RS) camera, we propose a dual Blur-RS setting to solve the motion ambiguity of blur decomposition. As shown in the Figure below, the RS view not only provides local details but also implicitly captures the temporal order of latent frames. Meanwhile, GS view could be exploited to mitigate the initial-state ambiguity from RS counterpart.

Dependencies

1. Python and Pytorch

• Pyhotn=3.8 (Anaconda recommended)
• Pytorch=1.11.0
• CUDA=11.3/11.4

conda create -n dualbr python=3.8
conda activate dualbr
conda install pytorch==1.11.0 torchvision==0.12.0 torchaudio==0.11.0 cudatoolkit=11.3 -c pytorch

2. Other packages

pip install -r requirements.txt

Data and Pretrained Model

• Download datasets realBR and synthetic data GOPRO-VFI_copy based on GOPRO.
• Unzip them under a specified directory by yourself.
• Please download checkpoints from this link and put them under root directory of this project.

Test

To test model, please run the command below:

bash ./test.sh       ### Please specify your data directory, output path in the script

Train

To train model, please run the command below:

bash ./train.sh       ### Please refer to the script for more info.

Acknowledgement

This project is implemented by partially referring to the code of work below:

• RIFE