FPS drops significantly when adding visualizations

[xkaraman]

First of all, thanks for this package. It seems to have a lot of potential for fast prototyping and visualizing.

When i open Zethus tab, i add my RobotModel visualization, frames (widget?) report stable 60 fps.
When i add Image type, it drops to 9-10 Fps. Moreover if i hide the visualization, it barely improves.

Only if i delete it, fps are back to 60. Same goes if i add a PointCloud type, but fps drops to 1. Again only if i delete it, not hide it, fps come back to normal (60 fps).

Note, jupyterlab-ros master was turned off by me and it's seem off in jupyter widget on the bottom part. I started it standalone on my terminal and launch rosbridge_websockets. Then i edided zethus configuration and set the port that rosbridge started.

Is there something i am doing wrong or a configuration that i have missed that decrease the overall fps by that much that is unusable?

[wolfv]

Hi, thanks for the kind words.
To be honest I don't know where the problem lies. Would need to do some investigation -- this might be related to the large data sizes of the image topic...

[wolfv]

I am cc'ing the original author of Zethus, @Chaitanya-Deep – maybe he knows what's going on :)

[xkaraman]

Ok thanks. Is this still maintained btw? Because i see it hasn't had any commits for almost a year.

[wolfv]

hey, yeah, there is not so much development happening currently. But we're still interested in keeping these things alive and will work on the Amphion widgets for jupyter in the coming months.

Any help is welcome!

[seadeep42]

Hi,

I'll need to look into the exact cause but at the moment, can guess some possible causes.

The image and pointcloud messages are decoded on the main thread in JS:
https://github.com/RoboStack/amphion/blob/master/src/utils/pcl.ts#L56 and https://github.com/RoboStack/amphion/blob/master/src/utils/processing.ts#L144

We tried to do it in wasm but there were some problems at the time with rollup build in Amphion and webpack in Zethus. We didn't get time to fix it:
https://github.com/RoboStack/amphion/blob/master/src/utils/attachPCL.js#L8

Hiding the viz just removes it in the UI but keeps the instance around. This is to ensure if a new message isn't received on the topic but the user unhides it, it must show the most recent data received on the topic.

[xkaraman]

Thanks for the quick response here!! :) So Amphion will be a replacement of Zethus or it provides the functionality to Zethus? I am bit confused with the packages.

. This is to ensure if a new message isn't received on the topic but the user unhides it, it must show the most recent data received on the topic.

Why is that a requirement? If my camera somehow shutdown, displaying the most recent data would be misleading. In ROS there is parameter named latched in Publisher code i believe that essentially does this but is for mostly static or low-rate changing data. See parameter latched

[seadeep42]

Amphion provides the functionality for Zethus. It handles subscribing and unsubscribing from topics. It's meant to be easily extensible. Web developers could easily add it to 3D scenes that also have things apart from ROS visualizations.

The latched parameter makes sense. I can update it to unsubscribe on hiding the viz but will have to wait till the weekend for the change. Do you have a rosbag I can use to test it?

[xkaraman]

Yea it's not urgent. Just trying things out here. I can create just small rosbag if that's all you need with minor movements, images and a latched tf_static and probably some other topics.

[seadeep42]

Yes. A small rosbag will be great. I can play it in a loop. Preferably the image and point cloud topics so I can try to improve the visualizations too. Thank you

[xkaraman]

Cool i created a small duration bag ~3 seconds. It contains /tf /tf_static /camera/color/image_raw /camera/color/image_raw/compressed /camera/depth/points. In order to view pointcloud, frame should be set to base_link or base_footprint since there is no map or world coordinate frame.

WeTransfer Link