Skip to content

Self-supervised learning method for motion prediction

Notifications You must be signed in to change notification settings

KIT-MRT/road-barlow-twins

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Road Barlow Twins: Redundancy Reduction for Motion Prediction

TL;DR: New self-supervised learning method for motion prediction in self-driving applications.

Overview

Overview

Road Barlow Twins. During pre-training, plain map data is used. The pre-training objective is to learn similar embeddings for differently augmented views of the same map data. During fine-tuning, annotated samples with past traffic agent trajectories are used to fine-tune for motion prediction.

Follow-up work

We continued this work with a focus on learned token set representations instead of rasterized image representations in this repository fork.

Prepare waymo open motion prediction dataset

Register and download the dataset from here. Clone this repo and use the prerender script as described in the readme.

Acknowledgements

The local attention (Beltagy et al., 2020) and cross-attention (Chen et al., 2021) implementations are from lucidrain's vit_pytorch library. The baseline DualMotionViT model builds upon the work by Konev et al., 2022.

About

Self-supervised learning method for motion prediction

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages