one_d_rpm breaks with different masses for HoverAviary

[KevinHan1209]

Hi,

I am running into an odd issue where if I change the mass by a small amount, say 0.05 kg, using the pybullet.changeDynamics() method in the _housekeeping() method, the training for one_d_rpm gets stuck at a very small reward and never improves. In some cases, during the rendering/evaluation it just oscillates above and below 1m. I have no idea why this is. It seems that one_d_pid does not have this issue and works well. I'm also slightly confused about the 5% parameter when updating RPMs in relation to the hover RPM. Does anyone have any experience with this?

[abdul-mannan-khan]

Yes. There is a bug in this HoverAviary.py. It only works for the given example. You make small changes like mass, or action or even goal point, it will not work. Particularly, if you change goal point from (0,0,1) to (1,0,1), it will not work. I guess you have to modify reward function.

[KevinHan1209]

I don't think it's the policy or the environment (reward function) that's the problem but rather the simulation settings itself. It doesn't make sense that a reward function would suddenly fail with marginal changes in mass. I'm suspecting the PyBullet physics implementation may be wrong. I saw that the thrust2weight ratio was also set as a constant in the urdf file, which doesn't make sense to me either.

[abdul-mannan-khan]

I agree with you. I am also facing the same problem. I think it would be much better to make own environment using ChatGPT. I have realized that physics and low level controller does not work properly particularly one action is rpm or pid instead of one_d_rpm or one_d_pid.

[JacopoPan]

Two main clarifications:

• @abdul-mannan-khan the one_d action spaces limit actuation to only vertical thrust so, of course, cannot be used to move in x-y
• @KevinHan1209 if you want to change the physical properties of the drone, the way I structured the code, you should do it in the robot description file (.URDF). The issue you are facing is almost certainly created by the +-5% interval because it's around the self.HOVER_RPM calculated from the thrust necessary to beat gravity, calculated from the mass in the URDF. (the reason to limit the action space to that 10% of possible RPMs is because of course only a small part of the range 0-20000+ that the motors are capable of is actually useful for leveled flight)

Hope this helps, btw, if you are looking for an environment where to randomize mass and inertia, we created safe-control-gym specifically for that.