Optimize what the realsense cameras are doing

[mjsobrep]

Right now the realsense cameras are publishing a lot, they take up something like 30% of the CPU. It has to be possible to reduce the load. Maybe there are things we don't need. The first candidate would be point clouds. We can always reconstruct point clouds later if we need to, right? This idea needs testing and experimentation

[mjsobrep]

Do we need to get the IR feeds? we definitely want to get depth. It might be worth having just the one left IR camera, don't think we need both. But I can't tell if we can get depth without having them on.

[mjsobrep]

apparently aligning depth to color is also taxing. This is another thing that we should be able to do later given the data. <- need to test

[mjsobrep]

Compliling without OpenMP might improve things: IntelRealSense/librealsense#1130

[mjsobrep]

Part of the problem was that it was publishing and therefore transmitting over USB the infrared streams. That was fixed in #28

[mjsobrep]

Ok, so we mostly need to test that the data we are getting is enough to reconstruct point clouds, align the depth to the color, etc.

[mjsobrep]

Ok, so there are a few options for recording and reconstructing the data later:

• Use the built in record to bag file in the realsense api -> This does no compression, you have no control. Files are too big and aren't with the reset of the data.
• Record to a regular bag file and try to use the other image pipelines from the open source stuff to do reconstruction. @suveergarg is trying that out.
• Create a software device and inject the images from the bag file in. Then we can easily use the realsense processing tools. This has the benefit of making the other realsense tools available too. There is some discussion of doing that Align depth to color IntelRealSense/librealsense#2534 and How to get the depth value corresponding to the color in the depth image? IntelRealSense/librealsense#2231
• Could pull in the image pairs as a frameset and align in python (https://github.com/IntelRealSense/librealsense/blob/jupyter/notebooks/distance_to_object.ipynb). Would need to make a frameset possiby by passing in a frame. There is discussion on doing that Creating source frames IntelRealSense/librealsense#956.

After going through all of this. I think that the way to go is to inject the frames into a synthetic software device and then process from there. I think this can be done in either python or C++. Can manually integrate with ROS.