August 2019
tl;dr: Predicting coordinate transformation (predicting x and y directly from image and vice versa) with Conv Nets are hard. Adding a mesh grid to input image helps this task significantly.
The paper results are very convincing, and the technique is super efficient. Essentially it only concats two channel meshgrid to the original input.
RoI10D cited this paper. This work also inspired cam conv.
- Other coordinates works as well, such as radius and theta.
- The idea can be useful for other tasks such as object detection, GAN, DRL, but not so much for classification.
- Summary of technical details
- Uber made a video presenting this paper.
- A concurrent paper from VGG has more theoretical analysis Semi-convolutional Operators for Instance Segmentation ECCV 2018.
- This technique seems to alleviate checkerboard artifacts as well. This is a good alternative to using bilinear upsampling.