Different intrinsic matrix values for rectified left and right lens

[kushal2507]

Check if issue already exists

• Google it (e.g. error xy github luxonis depthai)
• Check troubleshooting in documentation.

Describe the bug
A clear and concise description of what the bug is.

Minimal Reproducible Example
Append the MRE to the bug report, instructions here

If available launch files don't work in your case, please check if you also get errors while running:

• stereo_inertial_node in depthai_examples
• camera in depthai_ros_driver
• In case both fail to run, please check if you can run the default python demo app

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Pipeline Graph

Please also provide a screenshot of your pipeline using the DepthAI Pipeline Graph.

You can save it in depthai_ros_driver, either by calling /save_pipeline ROS service, or by setting parameter camera.i_pipeline_dump in ROS 2 or camera_i_pipeline_dump in ROS. Pipeline dump is saved to /tmp/pipeline.json.

Attach system log

• Provide output of log_system_information.py
• Which OS/OS version are you using?
• Which ROS version are you using?
• Which ROS distribution are you using ?
• Is depthai-ros built from source or installed from apt?
• Is depthai/depthai-core library installed from rosdep or manually? For rosdep install, check if ros-<rosdistro>-depthai package is installed, manual install can be checked with ldconfig -p | grep depthai
• Please include versions of following packages - apt show ros-$ROS_DISTRO-depthai ros-$ROS_DISTRO-depthai-ros ros-$ROS_DISTRO-depthai-bridge ros-$ROS_DISTRO-depthai-ros-msgs ros-$ROS_DISTRO-depthai-ros-driver
• To get additional logs, set DEPTHAI_DEBUG=1 and paste the logs, either from command line or from latest log in ~/.ros/log

Additional context
Add any other context about the problem here.

Package versions :

I am using the OAK-D-S2 ON Ubuntu 22.04, ROS2 Humble.

This is my camera.yaml file :

/oak:
ros__parameters:
camera:
i_mx_id: ''
i_usb_port_id: ''
i_calibration_dump: false
i_nn_type: none
i_enable_imu: true
i_pipeline_type: RGBD
i_usb_speed: SUPER_PLUS
i_restart_on_diagnostics_error: false
imu:
i_enable_rotation: true
i_rot_freq: 200
rgb:
i_fps: 12.0
i_resolution: 1080P
i_set_isp_scale: false
stereo:
i_align_depth: true
i_publish_right_rect: true
i_publish_left_rect: true
i_publish_synced_rect_pair: true
i_publish_topic: true
left:
i_fps: 12.0
right:
i_fps: 12.0

---

I am using the camera.launch.py file as is from the github repo Link

---

Upon echoing the left and right camera_info topics I noticed that the intrinsic matrix values are different for both lenses. Ideally, for rectified images shouldn't the values be the same? Attaching the outputs below.

Another issue is that on rviz the rectified images appear to have an irregular black border as seen below

Let me know if any additional information is needed

[Serafadam]

Hi, those values correspond to unrectified images (IIRC some libraries require this original information). Regarding black borders, this is expected due to rectification and the amount of distortion present in unrectified camera.

[kushal2507]

thanks for your reply

1. So just to clarify, in order for me to access the "rectified" intrinsic parameters for the left and right lens, I should refer to the projection matrix in the camera_info topic?

2. I am actually viewing the rectified camera streams in the images, so I would expect those to be free of distortion, right?

I am particularly interested in these 2 topics because I am using the rectified images and corresponding camera_info to calculate the depth/disparity myself.