[Feature] BEVFUSION radar input modality integration

[gorkemguzeler]

What is the feature?

here is the radar branch integration of bevfusion project:
mit-han-lab/bevfusion@d0152cf

As it is not officially supported by mmdet, I have spent some time in order to integrate the radar modality into my local repository. Some key takeaways, if you want to do it yourself:

• you will need radar information in the pickle files, therefore you need to process raw nuscenes dataset again with this additionaly information. for this step refer to the min-han-lab's pickle generator scripts.
• you will be able to generate the info pickles, but they will be in the previous version that mmdet used to work with.
• you need to update these pickles; but the update_infos script from mmdet does not support radar keys. so you need to update this script a bit (not much, just map radar information to new place)
• then you will finally have the pickles that contains radar information - also compatible to mmdet's current version.
• now, you should add/change some files to do the data processing for radar correctly (radar points, loading, formatting, Det3DDataPreprocessor etc.). for some of these files, you could refer to the mit-han-lab, but as the scripts are based on older version - you could get some issues.

As my codes are not well structured and contains some hard coded lines with private info - i cannot share that. However, following the above steps will save a lot of time until official implementation is done.

Hope it helps!

Any other context?

Alternatively, you could go to original repository of bevfusion to work with, but it is not maintained any more :/