Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[realsense_splitter_node]: Realsense frame metadata did not contain "frame_emitter_mode". Splitter will not work. #120

Open
akupferb opened this issue Aug 21, 2024 · 8 comments
Open

[realsense_splitter_node]: Realsense frame metadata did not contain "frame_emitter_mode". Splitter will not work. #120

akupferb opened this issue Aug 21, 2024 · 8 comments

Comments

@akupferb
Copy link

akupferb commented Aug 21, 2024 •
edited
Loading

My config for the realsense camera:

depth_module:
        emitter_always_on: False
        emitter_enabled: 1
        emitter_on_off: True

based on the newest version from realsense-ros

My remaps are all aligned:

                    ('input/depth',            front_cam_topic+'/depth/image_rect_raw'),
                    ('input/depth_metadata',   front_cam_topic+'/depth/metadata'),
                    ('input/infra_1',          front_cam_topic+'/infra1/image_rect_raw'),
                    ('input/infra_1_metadata', front_cam_topic+'/infra1/metadata'),
                    ('input/infra_2',          front_cam_topic+'/infra2/image_rect_raw'),
                    ('input/infra_2_metadata', front_cam_topic+'/infra2/metadata'),

I am getting this error when the infra streams are enabled:
[realsense_splitter_node]: Realsense frame metadata did not contain "frame_emitter_mode". Splitter will not work.
@alexmillane
Copy link
Collaborator

alexmillane commented Aug 21, 2024 •
edited by jaiveersinghNV
Loading

Please switch to the 3 versions of realsense software stated here. We're relying on the realsense software versions being tightly specified.

@akupferb
Copy link
Author

Note, the link you've attached does not lead to any page... I get a "server not found" error.
I found the correct(?) page here: https://nvidia-isaac-ros.github.io/getting_started/hardware_setup/sensors/realsense_setup.html

@akupferb
Copy link
Author

akupferb commented Sep 3, 2024

Hi @alexmillane

I've updated my versions to match those required by Isaac ROS. I did not change my Realsense firmware version (that was still 5.13.0.50), nor the librealsense SDK version, only needing to downgrade my realsense-ros version from the recent v4.55.1 release.
I will note that the specific combination of librealsense SDK and realsense-ROS versions are not officially supported by Intel according to their Releases. The realsense-ROS version 4.51.1 as required by IsaacROS corresponds to SDK v2.51.1, not v2.55.1.
Related to this, I get the following warning (and sometimes error) message when running the realsense node with the given versions:

{...}: RealSense ROS v4.51.1
{...}: Built with LibRealSense v2.55.1
{...}
{...} WARNING [140530306508352] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
{...}
{...} WARNING [140530306508352] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11

This error was not seen when running realsense-ROS version 4.55+ with the SDK v2.55.1

Is there any plan to update the topics/metadata handling in the isaac ros nodes to align with Intel's newer releases, which include updates and bug fixes, such that we could utilize the package versions that are officially supported together?

@alexmillane
Copy link
Collaborator

alexmillane commented Sep 24, 2024 •
edited
Loading

Did aligning the version to those I suggested (modulo my incorrect link) solve this original issue?

I have also seen that warning. From memory, things still operate correctly, in my experience, in spite of it.

Regarding upgrading our support to bleeding edge realsense software: we spend a surprising amount of time testing realsense versions and trying to upgrade. In general, our release cycles are not aligned (this makes sense, we're at different companies), so it is usually the case that Realsense has released a more recent version, following the latest Isaac release. Furthermore, the Realsense team tends to make interface changes, and we're limited in the amount of time we're able to spend playing that particular game of catch-up.

@akupferb
Copy link
Author

Yes, thank you, the original issue was solved. I also see that things operate correctly (for the most part) in spite of the error messages.

Thanks for the details on the version support, those limitations makes sense.

@akupferb
Copy link
Author

Hi @alexmillane
My mistake, the original issue has not been solved. I am still getting the "Realsense frame metadata did not contain "frame_emitter_mode". Splitter will not work. error with the versions set to the listed supported ones.... I have triple checked my topics and remappings and they are correctly assigned

In addition, while now running with the listed versions, I am getting repeated instances of the kernel-related error below:

[localization_container-3] [INFO] [1727610745.027613881] [nav_cam.rs_front]: RealSense ROS v4.51.1
[localization_container-3] [INFO] [1727610745.027715195] [nav_cam.rs_front]: Built with LibRealSense v2.55.1
[localization_container-3] [INFO] [1727610745.027749980] [nav_cam.rs_front]: Running with LibRealSense v2.55.1
[localization_container-3] [WARN] [1727610745.104514870] [nav_cam.rs_front]: No RealSense devices were found!
[...]
[localization_container-3] [INFO] [1727610751.874586036] [nav_cam.rs_front]: Device with serial number 102422070482 was found.
[localization_container-3] 
[localization_container-3] [INFO] [1727610751.874696822] [nav_cam.rs_front]: Device with physical ID /sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0/video4linux/video0 was found.
[localization_container-3] [INFO] [1727610751.874731319] [nav_cam.rs_front]: Device with name Intel RealSense D435 was found.
[localization_container-3] [INFO] [1727610751.875213825] [nav_cam.rs_front]: Device with port number 2-1.4.2 was found.
[localization_container-3] [INFO] [1727610751.875263811] [nav_cam.rs_front]: Device USB type: 3.2
[localization_container-3] [INFO] [1727610751.887364232] [nav_cam.rs_front]: getParameters...
[localization_container-3]  29/09 14:52:32,144 ERROR [281472800647392] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Invalid or incomplete multibyte or wide character
[localization_container-3]  29/09 14:52:32,144 ERROR [281472800647392] (rs.cpp:237) [rs2_load_json( dev:0xffff78018be0, json_content:0xffff78035790, content_size:3432 ) Backend] set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Invalid or incomplete multibyte or wide character
[localization_container-3] Failed to start device: set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Invalid or incomplete multibyte or wide character
[localization_container-3] [INFO] [1727610752.241938481] [nav_cam.rs_front]: Device with serial number 102422070482 was found.
[localization_container-3] 
[localization_container-3] [INFO] [1727610752.242054899] [nav_cam.rs_front]: Device with physical ID /sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0/video4linux/video0 was found.
[localization_container-3] [INFO] [1727610752.242086996] [nav_cam.rs_front]: Device with name Intel RealSense D435 was found.
[localization_container-3] [INFO] [1727610752.242439867] [nav_cam.rs_front]: Device with port number 2-1.4.2 was found.
[localization_container-3] [INFO] [1727610752.242479708] [nav_cam.rs_front]: Device USB type: 3.2
[localization_container-3] [INFO] [1727610752.249124588] [nav_cam.rs_front]: getParameters...
[localization_container-3]  29/09 14:52:32,495 ERROR [281472800647392] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Invalid or incomplete multibyte or wide character
[localization_container-3]  29/09 14:52:32,495 ERROR [281472800647392] (rs.cpp:237) [rs2_load_json( dev:0xffff780464b0, json_content:0xffff78003570, content_size:3432 ) Backend] set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Invalid or incomplete multibyte or wide character
[localization_container-3] Failed to start device: set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Invalid or incomplete multibyte or wide character
[localization_container-3] [INFO] [1727610752.595097947] [nav_cam.rs_front]: Device with serial number 102422070482 was found.
[localization_container-3] 
[localization_container-3] [INFO] [1727610752.595216701] [nav_cam.rs_front]: Device with physical ID /sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0/video4linux/video0 was found.
[localization_container-3] [INFO] [1727610752.595253022] [nav_cam.rs_front]: Device with name Intel RealSense D435 was found.
[localization_container-3] [INFO] [1727610752.595732520] [nav_cam.rs_front]: Device with port number 2-1.4.2 was found.
[localization_container-3] [INFO] [1727610752.595778825] [nav_cam.rs_front]: Device USB type: 3.2
[localization_container-3] [INFO] [1727610752.602196500] [nav_cam.rs_front]: getParameters...
[localization_container-3]  29/09 14:52:32,646 ERROR [281472800647392] (librealsense-exception.h:52) xioctl(VIDIOC_G_CTRL) failed Last Error: Protocol error
[localization_container-3]  29/09 14:52:32,646 ERROR [281472800647392] (rs.cpp:237) [rs2_load_json( dev:0xffff78029c50, json_content:0xffff78053e10, content_size:3432 ) Backend] xioctl(VIDIOC_G_CTRL) failed Last Error: Protocol error
[localization_container-3] Failed to start device: xioctl(VIDIOC_G_CTRL) failed Last Error: Protocol error
[localization_container-3]  29/09 14:52:32,648 ERROR [281472800647392] (librealsense-exception.h:52) set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Protocol error
[localization_container-3]  29/09 14:52:32,649 ERROR [281472800647392] (rs.cpp:237) [rs2_hardware_reset( device:0xffff78029c50 ) Backend] set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Protocol error
[localization_container-3] [ERROR] [1727610752.649563891] [nav_cam.rs_front]: Error starting device: set_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: Protocol error
[localization_container-3] [ERROR] [1727610752.860559391] [nav_cam.rs_front]: The device has been disconnected!
[localization_container-3] [INFO] [1727610753.730001909] [nav_cam.rs_front]: Checking new devices...
[localization_container-3] [INFO] [1727610753.810236826] [nav_cam.rs_front]: Device with serial number 102422070482 was found.
[localization_container-3] 
[localization_container-3] [INFO] [1727610753.810341820] [nav_cam.rs_front]: Device with physical ID /sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0/video4linux/video0 was found.
[localization_container-3] [INFO] [1727610753.810375037] [nav_cam.rs_front]: Device with name Intel RealSense D435 was found.
[localization_container-3] [INFO] [1727610753.810750949] [nav_cam.rs_front]: Device with port number 2-1.4.2 was found.
[localization_container-3] [INFO] [1727610753.810793222] [nav_cam.rs_front]: Device USB type: 3.2
[localization_container-3] [INFO] [1727610753.816515937] [nav_cam.rs_front]: getParameters...
[localization_container-3] [INFO] [1727610754.190533070] [nav_cam.rs_front]: JSON file is loaded! (/opt/ros/humble/share/squid_bringup/config/realsense/HighAccuracyFromViewer.json)
[localization_container-3] [INFO] [1727610754.190667601] [nav_cam.rs_front]: Device Name: Intel RealSense D435
[localization_container-3] [INFO] [1727610754.190705617] [nav_cam.rs_front]: Device Serial No: 102422070482
[localization_container-3] [INFO] [1727610754.190735122] [nav_cam.rs_front]: Device physical port: /sys/devices/platform/bus@0/3610000.usb/usb2/2-1/2-1.4/2-1.4.2/2-1.4.2:1.0/video4linux/video0
[localization_container-3] [INFO] [1727610754.190764979] [nav_cam.rs_front]: Device FW version: 5.13.0.50
[localization_container-3] [INFO] [1727610754.190790067] [nav_cam.rs_front]: Device Product ID: 0x0B07
[localization_container-3] [INFO] [1727610754.190813236] [nav_cam.rs_front]: Sync Mode: Off
[localization_container-3] [WARN] [1727610754.339286362] [nav_cam.rs_front]: re-enable the stream for the change to take effect.
[localization_container-3] [WARN] [1727610754.342874567] [nav_cam.rs_front]: re-enable the stream for the change to take effect.
[localization_container-3] [WARN] [1727610754.349142670] [nav_cam.rs_front]: re-enable the stream for the change to take effect.
[localization_container-3] [WARN] [1727610754.376866597] [nav_cam.rs_front]: Could not set param: rgb_camera.power_line_frequency with 3 Range: [0, 2]: parameter 'rgb_camera.power_line_frequency' could not be set: Parameter {rgb_camera.power_line_frequency} doesn't comply with integer range.
[localization_container-3] [WARN] [1727610754.389033580] [nav_cam.rs_front]: re-enable the stream for the change to take effect.
[localization_container-3] [WARN] [1727610754.389902079] [nav_cam.rs_front]: re-enable the stream for the change to take effect.
[localization_container-3] [WARN] [1727610754.392052557] [nav_cam.rs_front]: re-enable the stream for the change to take effect.
[localization_container-3] [INFO] [1727610754.469977280] [nav_cam.rs_front]: Stopping Sensor: RGB Camera
[localization_container-3] [INFO] [1727610754.514644388] [nav_cam.rs_front]: Starting Sensor: RGB Camera
[localization_container-3] [INFO] [1727610754.525094118] [nav_cam.rs_front]: Open profile: stream_type: Color(0), Format: RGB8, Width: 640, Height: 480, FPS: 15
[localization_container-3] [INFO] [1727610754.530013936] [nav_cam.rs_front]: RealSense Node Is Up!

@alexmillane
Copy link
Collaborator

Hmm strange. Are you running inside the isaac ROS docker?

@akupferb
Copy link
Author

In the Docker container, yes; a modified version of the Isaac ros docker that includes all its required dependencies and installations but also our own robot software.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@alexmillane @akupferb