Having parallel camera support

[lukicdarkoo]

I have just checked the roadmap and there is no a parallel camera support.

I believe that the parallel camera support (and an example) would be very useful for an image based DRL.

[Mayankm96]

Thank you for pointing this out!

We agree that multi-camera support is crucial to make full use of photo-realistic scenes for robot learning, and are actively working on this feature. It is currently hard to pin down the exact timeline by which this feature would be ready for release in Isaac Sim.

Till then, we can perhaps extend the existing camera class to have a multi-camera API to try out (as an experimental feature). What do you think?

[lukicdarkoo]

Sounds good! Ideally, the parallel camera API and the multicamera API would be similar so we can switch to the parallel camera API once it is ready.

[sashwat-mahalingam]

Does this parallelism refer to parallel camera data retrieval of one camera across environment instances, or parallel camera data retrieval of multiple cameras within one environment instance?

[Mayankm96]

We have extended the Camera class to provide a batch of data for images. Currently, it iterates over the camera in each environment instance and stores their data in a batch-first order.

However, this does not necessarily mean that rendering happens in parallel. We are working with the Isaac Sim team on getting tiled-rendering support for faster rendering times.

If you only care about static meshes and depth images, I suggest looking into the RayCastCamera class which uses Warp.