Adds possibility to initialize cameras with their intrinsic matrix

[pascal-roth]

Description

This PR adds the possibility of initializing cameras (both Raycaster Cameras and USD Cameras) with the intrinsic matrix instead of using the aperture parameters. The intrinsic matrix is defined in the pattern config and the pinhole cameras spawn config, respectively.

Moreover, it fixes the bug that the vertical aperture is not adjusted for the USD camera case (it will always stay at the default value, even if the horizontal aperture is set). The default is squared pixels; however, it also allows the setting of other values.

Fixes https://github.com/isaac-orbit/IsaacLab/issues/226

Type of change

• New feature (non-breaking change which adds functionality)

Checklist

• I have run the pre-commit checks with ./isaaclab.sh --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I have updated the changelog and the corresponding version in the extension's config/extension.toml file
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[pascal-roth]

@Mayankm96 I remember there were issues when the horizontal and vertical apertures differed for the USD camera case. Do you know if that has been resolved by now? Just from looking at the resulting image, I couldn't see something wrong, but otherwise, I will check again.

[Mayankm96]

Do these now work properly from rendering PoV? I don't know if this issue is resolved from Omniverse side. Also vertical and horizontal aperture should be same for "square" pixels?

[pascal-roth]

I don't know if they work properly. Will implement a test to see if the results between between raycaster and USD camera are the same if I set them

[pascal-roth]

apparently it is not resolved

[pascal-roth]

offesets are not resolved

[pascal-roth]

I enabled that we can set the vertical aperture, mainly because if we change the horizontal aperture in the USD camera case, the vertical one would always stay at the default value of 15.

Weirdly, the offsets seem to make a difference for both cameras but at a very different scale. I added a test to the raycaster camera that initializes both cameras with the same intrinsic matrix. If the offsets are enabled the values changes as follows:

offsets equal to 0

cam_warp_output
tensor([[[0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
         [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
         [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
         ...,
         [3.2779, 3.2779, 3.2779,  ..., 3.2778, 3.2778, 3.2778],
         [3.2735, 3.2735, 3.2735,  ..., 3.2734, 3.2734, 3.2734],
         [3.2692, 3.2692, 3.2692,  ..., 3.2691, 3.2691, 3.2691]]],
       device='cuda:0')
cam_usd_output
tensor([[[0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
         [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
         [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
         ...,
         [3.2757, 3.2757, 3.2757,  ..., 3.2756, 3.2756, 3.2756],
         [3.2713, 3.2713, 3.2713,  ..., 3.2712, 3.2712, 3.2712],
         [3.2670, 3.2670, 3.2670,  ..., 3.2669, 3.2669, 3.2669]]],
       device='cuda:0')

offsets unequal 0

cam_warp_output
tensor([[[0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
         [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
         [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
         ...,
         [3.2435, 3.2435, 3.2435,  ..., 3.2434, 3.2434, 3.2434],
         [3.2393, 3.2393, 3.2393,  ..., 3.2392, 3.2392, 3.2392],
         [3.2350, 3.2350, 3.2350,  ..., 3.2349, 3.2349, 3.2349]]],
       device='cuda:0')
cam_usd_output
tensor([[[0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
         [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
         [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
         ...,
         [3.2748, 3.2748, 3.2748,  ..., 3.2747, 3.2747, 3.2747],
         [3.2704, 3.2704, 3.2704,  ..., 3.2703, 3.2703, 3.2703],
         [3.2661, 3.2661, 3.2661,  ..., 3.2660, 3.2660, 3.2660]]],
       device='cuda:0')

@Dhoeller19 and @Mayankm96, do you know how the offsets are handled internally for the USD camera? IMO we handle them correctly for the raycaster case, so that I added warnings now in the USD camera that these values are basically ignored