How to use this package in ROS Kinetic+UR5+KinectV2 ?

[UyaSong]

Hi all,
I'm a new user of ROS and I'm trying to use this package in my UR5+Kinect system.
In my system, I have already done the following works:
1.control the UR5 by ROS moveit and ur_modern_driver;
2.handeye calibration between UR5 and Kinect;
3.read the RGB, depth, and pointcloud of KinectV2 in Rviz;
4.transform depth or pointcloud into octomap to realize obstacle avoidance.
But I found it's really slow for step 4, so I'm trying to filter the robot and background in depth or pointcloud in order to focus on the dynamic obstacle part.
Luckly this package seems to provide such function. But there isn't quite much tutorial for using this great work. I have tried to change the realtime_urtf_filter.launch for my system but there is not the filtered depth output. I start these files for running this package:
1.roslaunch kinect2_bridge kinect2_bridge.launch
2.roslaunch ur_modern_driver ur5_bringup.launch
3.roslaunch ur5_moveit_config ur5_moveit_planning_execution.launch
4.roslaunch ur5_moveit_config moveit_rviz.launch
5.roslaunch realtime_urdf_filter realtime_urtf_filter.launch

The changed realtime_urtf_filter.launch file:
realtime_urdf_filter_plus.txt

mainly change the topic name

What else should I do to run this package? Any suggestion would be grateful

[UyaSong]

Hi,
The problem above has been solved. The cause of my problem is that I did not config the realtime_urtf_filter.launch correctly. I learned from this to change my .launch file. Thanks to @JimmyDaSilva
But another problem occurred. The Links in the filtered image seems not being in the right position, just like the image below:

It's quite strange. It seems the tf is working correctly because the robot model is right:

Any suggestion would be grateful!

[blodow]

Hi! Thanks to @JimmyDaSilva for helping out here!

I think the issue is that the urdf file is not being loaded correctly. Likely some transformation per link that is not parsed correctly. RViz is obviously loading the model file correctly. I see two options: either change the model file so it's loaded correctly (maybe it's possible to "bake" in transformations) or fix the loading code...

Unfortunately, I have not been doing robotics research in quite some time, so this project is somewhat dormant, as I'm sure you have noticed.

@JimmyDaSilva are you still actively using this project, and would you be interested in taking over maintenance?

[UyaSong]

@blodow Thanks to your reply and suggestion. I will take a try and let you know if it works.
Thanks very much for your work, it is useful and significant. It would be great if it can be maintained.

[JimmyDaSilva]

@SunYuansong Glad that my branches were of help.
Maybe we can discuss a little more by email if you need help for configuration.

As for the issue you are demonstrating, I never had this exact problem before.
Could you add a "Robot model" object on RVIZ to check if the robot is well loaded.
(The one showed here is a MoveIt one which could be just a plan)

@blodow I am still in research and robotics, but I haven't been playing with this package for 2 years now.
But I can definitely try to answer issues and PRs

[UyaSong]

@JimmyDaSilva Thanks to your reply and suggestion. I have tried to load the ur5 model in RVIZ and it can be shown correctly:

It seems the model is fine, maybe the way I load it has some issues?
Here is my filter.launch file:
filter_kinectV2.launch.txt
I'm using the ur5 model from universal_robot/ur_description/urdf/ur5_robot.urdf.xacro
Thanks for your kind help

[blodow]

i suspect that the model uses something that my renderer does not read correctly. e.g. look at

realtime_urdf_filter/src/urdf_renderer.cpp

Line 125 in 80473bd

urdf::Rotation rotation = link->visual->origin.rotation;

, where we process a single link from the URDF, specifically get its visual element and its pose. A link can have multiple entries, visual, collision etc., as can be seen here: https://wiki.ros.org/urdf/XML/link and maybe your model requires the renderer to use URDF features that were not required for other models so far. I would suggest debugging into that bit of code to see if there is relevant information stored in the link that is ignored by the code.

[zhulei2017]

Hi, @SunYuansong , I meet the same problem with robot UR5. I used this package by changing the “.dae” file of UR5 in in universal_robot
The default up axis in universal_robot is Z axis while the default up axis in realtime_urdf_filer package is Y axis . So I modified the “.dae” file with VS code and change the <up_axis>Z_UP</up_axis> into <up_axis>Y_UP</up_axis>. Then, it works well.
I hope that it also works for you. More information about dae file is introduced in this article

Thanks for @blodow and @JimmyDaSilva works. It helps a lot in my research.

[UyaSong]

@zhulei2017 Thanks very much! This bug is really hidden. I can get the expected output now:

[blodow]

Great! looking at this issue here, it seems that RViz simply ignores the up_axis tag, whereas assimp (used here) respects it. Since RViz likely won't change fix bug to ensure backwards compatibility, and i cannot find an assimp flag to read or ignore this tag, changing the file (or writing our own collada importer haha) is currently the most sensible "solution".

[UyaSong]

It explains why the rviz has a correct output. It can be a demonstration for the latter users.
Thank you all again, I will close this issue now.

[kingjin94]

For future reference: the same axis error (Z_UP instead of Y_UP) can also be found with the Franka Panda.
Furthermore I think this whole hassle is a assimp issue, as they have open issues on the handling of Z_UP from 2016/17 (e.g. assimp/assimp#849, assimp/assimp#1463)

[xianyitiantim]

Hi all, could somebody tell me how to use this package in noetic? Shall I launch the file in sequence as @UyaSong described in the question? But when I launch the fifth file, it would show:

[ WARN] [1624612198.066798493]: Shutdown request received.
[ WARN] [1624612198.069377257]: Reason given for shutdown: [[/robot_state_publisher] Reason: new node registered with same name]

I think this node has already been running in roslaunch ur_modern_driver ur5_bringup.launch, therefore it should not run again.

Also, shall I change show_gui to true in the yaml file? I have done this to make the gui appear.

When the gui pops up, it is very strange as it shows nothing.

Possible problem would be I do not meet the dependency. Actually I have install the glew 1.60, but when I type in glewinfo it just said that my glew version is 2.1.0

Can anyone help me? Thanks a lot!!