RAFT: Recurrent All Pairs Field Transforms for Optical Flow
We introduce Recurrent All-Pairs Field Transforms (RAFT), a new deep network architecture for optical flow. RAFT extracts perpixel features, builds multi-scale 4D correlation volumes for all pairs of pixels, and iteratively updates a flow field through a recurrent unit that performs lookups on the correlation volumes. RAFT achieves state- of-the-art performance. On KITTI, RAFT achieves an F1-all error of 5.10%, a 16% error reduction from the best published result (6.10%). On Sintel (final pass), RAFT obtains an end-point-error of 2.855 pixels, a 30% error reduction from the best published result (4.098 pixels). In addition, RAFT has strong cross-dataset generalization as well as high efficiency in inference time, training speed, and parameter count. Code is available at https://github.com/princeton-vl/RAFT.
| Models | Training datasets | Flying Chairs | Sintel(training) | KITTI2015(training) | Log | Config | Download | ||
| clean | final | Fl-all | EPE | ||||||
|---|---|---|---|---|---|---|---|---|---|
| RAFT | Flying Chairs | 0.80 | 2.27 | 4.85 | - | - | log | Config | Model |
| RAFT | FlyingChairs + FlyingThing3d | - | 1.38 | 2.79 | 16.23% | 4.95 | log | Config | Model |
| RAFT | FlyingChairs + FlyingThing3d + Sintel | - | 0.63 | 0.97 | - | - | log | Config | Model |
| RAFT | Mixed Dataset[1] | - | 0.63 | 1.01 | 5.68% | 1.59 | log | Config | Model |
| RAFT | KITTI2015 | - | - | - | 1.45% | 0.61 | log | Config | Model |
@inproceedings{teed2020raft,
title={Raft: Recurrent all-pairs field transforms for optical flow},
author={Teed, Zachary and Deng, Jia},
booktitle={European conference on computer vision},
pages={402--419},
year={2020},
organization={Springer}
}[1] The mixed dataset consisted of FlyingChairs, FlyingThing3d, Sintel, KITTI2015, and HD1K.