Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(autoware_image_projection_based_fusion): add mask cluster fusion node #9482

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

StepTurtle
Copy link
Contributor

Description

This PR introduces a new node for fusing instance segmentation results with clusters, assigning label values to the clusters. It serves as an alternative to the roi_cluster_fusion node, which relies on bounding boxes.

Following table shows results with old and new approach:

Node Parameters Result Video
roi_cluster_fusion only_allow_inside_cluster = false https://github.com/user-attachments/assets/63fbb81d-5ae5-4d28-8918-1763789afa81
roi_cluster_fusion only_allow_inside_cluster = true https://github.com/user-attachments/assets/c9f966d2-60b6-49b3-bcb4-88ae998be4a2
mask_cluster_fusion default https://github.com/user-attachments/assets/f450af0c-a6af-49d0-9b78-1b089f65f577

Related links

Parent Issue:

How was this PR tested?

Testers can follow these steps to test PR.

1) Download the pre-trained RTMDet model:

gdown https://drive.google.com/drive/folders/1P1JyMrstJIN9Y4F947nUkbKxQ2JDntgF?usp=drive_link -O ~/autoware_data/ --folder

2) For autoware use following PR:

3) For autoware_internal_msgs use following PR:

4) For autoware.universe use this and following PR:

5) Build your Autoware workspace with new changes.

6) Run e2e or logging simulator.

7) Update the topic parameters from launch/rtmdet.launch.xml and run the launch file

ros2 launch autoware_tensorrt_rtmdet rtmdet.launch.xml

8) Update the topic parameters from launch/mask_cluster_fusion.launch.xml and run the launch file

ros2 launch autoware_image_projection_based_fusion mask_cluster_fusion.launch.xml
  • You should able to see newly added label names for cluster from /perception/object_recognition/detection/clustering/camera_lidar_fusion/clusters topic.

Notes for reviewers

  • Documentation is not ready right now, it will be added ASAP.

Interface changes

None.

Effects on system behavior

None.

Signed-off-by: Barış Zeren <[email protected]>
Signed-off-by: Barış Zeren <[email protected]>
Signed-off-by: Barış Zeren <[email protected]>
Signed-off-by: Barış Zeren <[email protected]>
@StepTurtle StepTurtle added the component:perception Advanced sensor data processing and environment understanding. (auto-assigned) label Nov 27, 2024
@StepTurtle StepTurtle self-assigned this Nov 27, 2024
Copy link

github-actions bot commented Nov 27, 2024

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:perception Advanced sensor data processing and environment understanding. (auto-assigned) tag:require-cuda-build-and-test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant