[BUG] {camera.launch.py not working}

[Mia-estudiante]

Describe the bug
I can't get topics related to depth image. Here is the bug report when launching ros2 launch depthai_ros_driver camera.launch.py
It is the similar error with ros2 launch turtlebot4_bringup oakd.launch.py

Before the node is down, I can see oak topics with rgb and stereo in topic lists(I attached the picture below). I wanna use all the topics for extracting the data, so I think launching camera.launch.py is the solution but it is down showing an error message.

So, what do I do execute the code successfully?
Please help.

and I have some questions.
Q1.
In an existing camera.yaml, there is no content related to rgb and stereo.
Then, what is the reason of showing topics related to them?
Q2.
When I read the official document, if I use RGBD as the value of i_pipeline_type, it says that I have to include left & right camera content.
But, an existing camera.yaml doesn't include left & right camera content.
Q3.
Can you tell me what the required parameters are if I modify the yaml file to customize topics?

System information
I am using ROS2 humble on Turtlebot4 (ubuntu 22.04).

Attach system log
[INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2024-04-09-06-21-36-602339-ubuntu-1579
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container-1]: process started with pid [1593]
[component_container-1] [INFO] [1712643700.753357558] [oak_container]: Load Library: /opt/ros/humble/lib/librobot_state_publisher_node.so
[component_container-1] [INFO] [1712643700.762664553] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate<robot_state_publisher::RobotStatePublisher>
[component_container-1] [INFO] [1712643700.762804922] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<robot_state_publisher::RobotStatePublisher>
[component_container-1] [INFO] [1712643700.841839772] [oak_state_publisher]: got segment oak
[component_container-1] [INFO] [1712643700.842024252] [oak_state_publisher]: got segment oak-d-base-frame
[component_container-1] [INFO] [1712643700.842071844] [oak_state_publisher]: got segment oak_imu_frame
[component_container-1] [INFO] [1712643700.842107548] [oak_state_publisher]: got segment oak_left_camera_frame
[component_container-1] [INFO] [1712643700.842139307] [oak_state_publisher]: got segment oak_left_camera_optical_frame
[component_container-1] [INFO] [1712643700.842169547] [oak_state_publisher]: got segment oak_model_origin
[component_container-1] [INFO] [1712643700.842197843] [oak_state_publisher]: got segment oak_rgb_camera_frame
[component_container-1] [INFO] [1712643700.842225880] [oak_state_publisher]: got segment oak_rgb_camera_optical_frame
[component_container-1] [INFO] [1712643700.842255732] [oak_state_publisher]: got segment oak_right_camera_frame
[component_container-1] [INFO] [1712643700.842283861] [oak_state_publisher]: got segment oak_right_camera_optical_frame
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/oak_state_publisher' in container 'oak_container'
[component_container-1] [INFO] [1712643700.865168112] [oak_container]: Load Library: /opt/ros/humble/lib/libdepthai_ros_driver.so
[component_container-1] [INFO] [1712643701.389025588] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate<depthai_ros_driver::Camera>
[component_container-1] [INFO] [1712643701.389210476] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<depthai_ros_driver::Camera>
[component_container-1] [INFO] [1712643701.513268889] [oak]: No ip/mxid specified, connecting to the next available device.
[component_container-1] [INFO] [1712643704.428852394] [oak]: Camera with MXID: 18443010F1D6990F00 and Name: 1.1.2 connected!
[component_container-1] [INFO] [1712643704.430665175] [oak]: USB SPEED: SUPER
[component_container-1] [INFO] [1712643704.478820471] [oak]: Device type: OAK-D-PRO
[component_container-1] [INFO] [1712643704.482657735] [oak]: Pipeline type: RGBD
[component_container-1] [INFO] [1712643704.840804483] [oak]: NN Family: mobilenet
[component_container-1] [INFO] [1712643705.228526574] [oak]: NN input size: 300 x 300. Resizing input image in case of different dimensions.
[component_container-1] [INFO] [1712643705.619172529] [oak]: Finished setting up pipeline.
[component_container-1] [18443010F1D6990F00] [1.1.2] [1712643709.256] [host] [warning] Device crashed, but no crash dump could be extracted.
[component_container-1] terminate called without an active exception
[ERROR] [component_container-1]: process has died [pid 1593, exit code -6, cmd '/opt/ros/humble/lib/rclcpp_components/component_container --ros-args --log-level info --ros-args -r __node:=oak_container -r __ns:=/'].

• List of topics checked before node is turned off
  

[Serafadam]

Hi, could you provide some more information as described in the bug report template?

Q1.
In an existing camera.yaml, there is no content related to rgb and stereo.
Then, what is the reason of showing topics related to them?

The driver's parameters all have some default values, previously they were all listed in the config files but that made them a bit hard to read and could cause some unexpected errors. You can see all the parameters here or you can list them in cli via ros2 param list /oak / ros2 param dump /oak

Q2.
When I read the official document, if I use RGBD as the value of i_pipeline_type, it says that I have to include left & right camera content.
But, an existing camera.yaml doesn't include left & right camera content.

These values can be just added to the config file as mentioned above.

Q3.
Can you tell me what the required parameters are if I modify the yaml file to customize topics?

Not sure what you exactly mean here.

[Mia-estudiante]

Thanks for response, and I provide more information.

Describe the bug
Running ros2 launch depthai_ros_driver camera.launch.py, the node is down.

Expected behavior
Running ros2 launch depthai_ros_driver camera.launch.py or ros2 launch turtlebot4_bringup oakd.launch.py, I expect topics below to extract the data.

/oak/rgb/camera_info
/oak/rgb/image_raw
/oak/rgb/image_raw/compressed
/oak/rgb/image_raw/compressedDepth
/oak/rgb/image_raw/theora
/oak/stereo/camera_info
/oak/stereo/image_raw
/oak/stereo/image_raw/compressed
/oak/stereo/image_raw/compressedDepth
/oak/stereo/image_raw/theora

Attach system log

• Provide output of log_system_information.py
• Which OS/OS version are you using? Ubuntu 22.04.4
• Which ROS version are you using? Ros2 Humble
• Which ROS distribution are you using ?
• Is depthai-ros built from source or installed from apt? built from source (git clone --branch humble https://github.com/luxonis/depthai-ros.git)
• Is depthai/depthai-core library installed from rosdep or manually? For rosdep install, check if ros-<rosdistro>-depthai package is installed, manual install can be checked with ldconfig -p | grep depthai
• Please include versions of following packages
  • Package: ros-humble-depthai Version: 2.24.0-1jammy.20240308.165854
  • Package: ros-humble-depthai-ros Version: 2.9.0-1jammy.20240327.182037
  • Package: ros-humble-depthai-bridge Version: 2.9.0-1jammy.20240327.165719
  • Package: ros-humble-depthai-ros-msgs Version: 2.9.0-1jammy.20240219.090343
  • Package: ros-humble-depthai-ros-driver Version: 2.9.0-1jammy.20240327.180744
• To get additional logs, set DEPTHAI_DEBUG=1 and paste the logs, either from command line or from latest log in ~/.ros/log

Additional context

Q3.
Can you tell me what the required parameters are if I modify the yaml file to customize topics?
A3.
Not sure what you exactly mean here.

I have referred to various issues to solve the error.
I noticed one solution of them( here ).
It said that in order to enable the stereo topic, the oakd_pro.yaml file needed to be modified by referring to the parameters. ( It is for launching ros2 launch turtlebot4_bringup oakd.launch.py )
As you know, there are many parameters.
So, if I have to modify the camera yaml file, I wonder what parameters should be included or removed, and what values should be set to.

On your advice, I execute ros2 param list /oak & ros2 param dump /oak before the node turns off.
In result, I cannot get anything.
When I run the command, it is not spouting out the results, and the command window is still.

[Serafadam]

Hi,
could you post debug logs after running the node? To do that, run export DEPTHAI_DEBUG=1 and then ros2 launch depthai_ros_driver camera.launch.py
As I can see, turtlebot4_bringup oadk.launch.py uses rgb_stereo_node.launch.py from depthai_examples package which uses a different driver and won't react to the same parameters. Though by default both of those should provide depth image.
Could you also try running the driver on your host PC just to make sure everything is OK with the camera?
Regarding parameters, to get depth you can set camera.i_pipeline_type: RGBD or Depth. When these pipelines are used, they dynamically load following sensor components:

• rgb
• stereo
• left
• right
  To set parameters on one of them, just refer to the documentation, by default left/right topics are disabled to save bandwidth, to enable them, you can just set for example left.i_publish_topic: true

[Mia-estudiante]

Thanks for response.
I provide debug logs below.

1. To do that, run export DEPTHAI_DEBUG=1 and then ros2 launch depthai_ros_driver camera.launch.py

The process stops with a message indicating that it is dead.

debug_log1.txt

2. As I can see, turtlebot4_bringup oadk.launch.py uses rgb_stereo_node.launch.py from depthai_examples package which uses a different driver and won't react to the same parameters.

There is no rgb_stereo_node.launch.py in oakd.launch.py and I found rgb_stereo_node.launch.py in oakd_example.launch.py.
So, I launch ros2 launch turtlebot4_bringup oadk_example.launch.py
The process dies the same as above.

debug_log2.txt

3. Could you also try running the driver on your host PC just to make sure everything is OK with the camera?

If I run the driver on the host PC, the driver can't recognize the camera.
As far as I know, shouldn't we run it on the TurtleBot 4 Raspberry Pi image?
You can see [oak]: No ip/mxid specified, connecting to the next available device [oak]: No available devices in the debug log.
Could you tell me the right way to use it, if I'm mistaken?

debug_log3.txt

4. Regarding parameters, to get depth you can set camera.i_pipeline_type: RGBD or Depth.

On your advice, I set the parameters' values launching the codes.
As you check the debug logs, it isn't adjusted by set values.

ros2 launch depthai_ros_driver camera.launch.py camera.i_pipeline_type:=RGBD left.i_publish_topic:=true right.i_publish_topic:=true
debug_log4.txt

ros2 launch depthai_ros_driver camera.launch.py camera.i_pipeline_type:=DEPTH left.i_publish_topic:=1 right.i_publish_topic:=1
debug_log5.txt

Lastly, I have a question about executing the OAK-D sensor.
As I know, before launching the files related to OAK-D sensor ( ex. turtlebot4_bringup or depthai_ros_driver pkgs ), I have to press 'Restart' or 'Install' in turtlebot4-setup, is it right?
I always press 'Restart' before executing launch files.
It's been working well for a while, but an error message suddenly pops up, and the sensor is not recognized. ( [oak]: No ip/mxid specified, connecting to the next available device [oak]: No available devices )

Thank you for taking the time.

[Mia-estudiante]

Hi, I tried connecting the sensor with Host PC directly, and it was a success.
I attached the result of log.
As you see, the node loades completely, and there were topics I expected.
success_log.txt

In the result, it is sure that the launch code and yaml files are not the point of the problem.
I think the problem is for networking between OAK-D sensor connected with Raspberry Pi image and Host PC.
In addition, I don't think it is a Wi-Fi issue at least because I kept checking when I first executed the Turtlebot4.

So, I don't know what the problem is.
If you grasp the problem, I would like to ask your opinion concerning this issue.
I look forward to hearing from you.
Thank you.

[Serafadam]

Hi, sorry for the delay, just to make sure, could you also test if you could run some python depthai examples? In general, if camera works on PC without issues then the issue might be somewhere on the turtlebot side, I would contact their team and verify if the setup is up to date.

[Mia-estudiante]

Hi, thanks for response!

I executed 'hello_world.py' in depthai_tutorials and here the result is.
I think there is no error.

After you contact the turtlebot team, I would appreciate it if you could share the answers.