Unified dataset generator

[Divelix]

I just thought, is it feasible to create a pipeline, where you won't need to mimic a specific dataset format (KITTI, Waymo , etc.) or write custom dataset class, but just specify paths to each type of data you have (pointclouds path, labels path, images path, calibration path, etc.) and require following just one rule: corresponding data for one sample has the same name - name of sample. For example, sample number 1 will fetch 1.bin (or any other supported extension for pointclouds) from pointclouds path (if it was specified), 1.png (or any other supported extension for images) from images path (if it was specified) and so on.

What drawbacks such approach has and how hard would it be to implement such feature?

[JingweiZhang12]

The specific dataset classes, such as KITTI, and Waymo, inherit from the parent class: Custom3DDataset, which is the common dataset generator. The reason why we implement the different specific dataset classes is that different datasets have different original annotation formats, and we need to process and unify them. Thus, we can use the common dataset generator.

[Divelix]

Sounds reasonable. This question came to me when I tried to make lidar only dataset/converter related code and faced so much complications, trying to understand how to do that based on KITTI related classes (which are rether complex due to multimodality). Probably, it would be nice to have something easier than KITTI to learn about mmdetection3d data pipelines (lidar only is a good candidate considering how much people are interested in it according to issues like #260).

[JingweiZhang12]

Thanks for your suggestions. Simplifying the dataset processing and data structures will be an item with high priority in the future.