Question: Low level control in RL environments

[maxspahn]

Hi everyone,

I was so happy to see the first public staff from HF on robotics here, because I am very keen on helping implementing some safety filter for RL policies, that can potentially be used for both training and testing. However, I realized a shortcut that some RL environments in sim do these days: Simply using a equality constraint on the end-effector to simplify the action space, see https://github.com/fyhMer/fowm as well. That however makes the transfer to the real-world difficult as most robots can only be controlled using joint positions or joint velocities. It seems rather unnatural to rely on the physics engine to do the mapping from Cartesian commands to joint space implicitly, rather than relying on an IK solver or an impedence scheme for that, which is also accessible in the real world.

What are your thoughts on this?

gym-xarm/gym_xarm/tasks/assets/shared.xml

Lines 36 to 38 in 415811f

	<equality>
	<weld body1="robot0:mocap2" body2="link7" solimp="0.9 0.95 0.001" solref="0.02 1"></weld>
	</equality>

As a counterexample, see https://github.com/google-deepmind/mujoco_menagerie/blob/main/aloha/aloha.xml which uses joint position control.