Jing Lin*, Yuanhao Cai*, Xiaowan Hu, Haoqian Wang, Youliang Yan, Xueyi Zou, Henghui Ding, Yulun Zhang, Radu Timofte, and Luc Van Gool
- 2022.08.05 : Pretrained model of FGST on GOPRO dataset is released. 🚀
- 2022.05.14 : Our FGST and S2SVR are accepted by ICML2022. 🚀
Super-Resolution | Deblur | Compressed Video Enhancement |
---|---|---|
- Flow-Guided Sparse Transformer for Video Deblurring (ICML 2022)
- Unsupervised Flow-Aligned Sequence-to-Sequence Learning for Video Restoration (ICML 2022)
pip install torchvision==0.9.0 torch==1.8.0 torchaudio==0.8.0
pip install -r requirements.txt
pip install openmim
mim install mmcv-full
pip install -v -e .
pip install cupy-cuda101==7.7.0
Download the datasets (GOPRO,DVD,REDS,VIMEO,MFQE-v2) and and recollect them as the following form:
|--VR-Baseline
|--data
|-- GoPro
|-- test
|-- train
|-- DVD
|-- test
|-- train
|-- REDS
|-- test
|-- train
|-- VIMEO
|-- test
|-- train
|-- MFQEV2
|-- test
|-- train
cd VR_Baseline
# training FGST
bash tools/dist_train.sh configs/FGST_deblur_gopro.py 8
# training S2SVR
bash tools/dist_train.sh configs/S2SVR_deblur_gopro.py 8
The training log, trained model will be available in VR-Baseline/experiments/
.
Download pretrained model and run the following command.
python demo/restoration_video_demo.py ${CONFIG} ${CHKPT} ${IN_PATH} ${OUT_PATH}
These works are mostly done during the internship at HUAWEI Noah's Ark Lab. Due to the limitation of company regulations, the original pre-trained models can not be transferred and published here. We will retrain more models and open-source them when we have enough GPUs as soon as possible.
- More Pretrained Models
- Inference Results
We refer to codes from BasicVSR++ and mmediting. Thanks for their awesome works.
If this repo helps you, please consider citing our works:
# FGST
@inproceedings{fgst,
title={Flow-Guided Sparse Transformer for Video Deblurring},
author={Lin, Jing and Cai, Yuanhao and Hu, Xiaowan and Wang, Haoqian and Yan, Youliang and Zou, Xueyi and Ding, Henghui and Zhang, Yulun and Timofte, Radu and Van Gool, Luc},
booktitle={ICML},
year={2022}
}
# S2SVR
@inproceedings{seq2seq,
title={Unsupervised Flow-Aligned Sequence-to-Sequence Learning for Video Restoration},
author={Lin, Jing and Hu, Xiaowan and Cai, Yuanhao and Wang, Haoqian and Yan, Youliang and Zou, Xueyi and Zhang, Yulun and Van Gool, Luc},
booktitle={ICML},
year={2022}
}