Skip to content

Latest commit

 

History

History
80 lines (58 loc) · 9.72 KB

README.md

File metadata and controls

80 lines (58 loc) · 9.72 KB
Raw

Gym pybullet environment

Example

  1. Useful links

    Universal robots repo

    OpenAI Gym repo

    OpenAI Robotic environments

    Pybullet Quickstart guide

    Pybullet Code (including gym examples)

  2. Preface

    This is a repository created for funzies to try different RL techniques using an UR5 robot (at this moment).

    So far, basic work has been done in setting up the environment. The test.py file in examples folder runs a basic loop, and gets state, reward and done info. I've written the initial UR5 reach environment in the most simple way possible for now to start any kind of loop. This project is written almost from scratch so there is a lot to add/improve.

  3. Installation

    Project is written using python 3!

    These installation instructions are for Linux, but I guess they could be run under Windows or OSX as well, for bullet3 SDK installation refer to the pybullet quickstart guide. I guess that is all that is needed outside the python packages.

    First you need bullet3 SDK for pybullet to work. Other than that, for now the only python packages required are gym, numpy and pybullet (I suggest using virtualenv). After getting those and cloning the folder you need to run python setup.py install in the main folder, and then pip install -e . (if this command doesn't work check if you didn't miss the dot, running this comand can also fix problem).

  4. TODO

    Since I just started this project, there is much to be done. For starters, rewards and states need to be defined and reformatted for usage with different RL algorithms.

    The first algorithm that I'll try to implement will be DDPG (I haven't yet decided if I'll use OpenAI's algorithms, or something simpler like this TORCS DDPG example). The robotic environments provide a lot to think about when it comes to the actions, for now I've decided to use velocity control, but position or 'delta' approaches can also be defined.

    Pybullet provides also an easy way to insert multiple robot models into one client, so that can also provide sone incentive to use them for something like genetic algorithms.

    Also, if the reach env will be completed, next environments could be defined (pick and place, or pushing some objects)

    For now, the plan is to:

    1. define rewards, states and targets
    2. add docs to the environment functions (most of the gym examples don't seem beginner friendly, at least to me)
    3. setup and run basic ddpg sim (if computer will allow this) using Tensorflow and maybe Keras

  5. Additional info

    UR5 URDF model used in simulation has been generated from xacro files present in UR5 description package (from github). Although it seems okay, if this project would be moved to a real robot I'd check if the information got agrees with real data. When starting the env there is some info about inertia values missing, but I don't want to bother with this now. Also, I've deleted the visual parts of the robot, since the shapes generated by pybullet seem clearer to me. Since the model is thrown without much thought into the environment, below is a simple return of the getJointInfo command from pybullet, to show joint names, indices and other info (very useful!).

getJointInfo legend:

name type description
jointIndex int the same joint index as the input parameter
jointName string the name of the joint, as specified in the URDF (or SDF etc) file
jointType int "type of the joint, this also implies the number of position and velocity variables. JOINT_REVOLUTE, JOINT_PRISMATIC, JOINT_SPHERICAL, JOINT_PLANAR, JOINT_FIXED. See the section on Base, Joint and Links for more details."
qIndex int the first position index in the positional state variables for this body
uIndex int the first velocity index in the velocity state variables for this body
flags int reserved
jointDamping float the joint damping value, as specified in the URDF file
jointFriction float the joint friction value, as specified in the URDF file
jointLowerLimit float Positional lower limit for slider and revolute (hinge) joints.
jointUpperLimit float Positional upper limit for slider and revolute joints. Values ignored in case upper limit smaller than lower limit..
jointMaxForce float Maximum force specified in URDF (possibly other file formats) Note that this value is not automatically used. You can use maxForce in 'setJointMotorControl2'.
jointMaxVelocity float Maximum velocity specified in URDF. Note that the maximum velocity is not used in actual motor control commands at the moment.
linkName string the name of the link, as specified in the URDF (or SDF etc.) file
jointAxis vec3 joint axis in local frame (ignored for JOINT_FIXED)
parentFramePos vec3 joint position in parent frame
parentFrameOrn vec3 joint orientation in parent frame
parentIndex int parent link index, -1 for base 
[(0,b'world_joint', 4, -1, -1, 0, 0.0, 0.0, 0.0, -1.0, 0.0, 0.0, b'base_link', (0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 1.0), -1),
 (1, b'shoulder_pan_joint', 0, 7, 6, 1, 0.0, 0.0, -6.283185307179586, 6.283185307179586, 150.0, 3.15, b'shoulder_link', (0.0, 0.0, 1.0), (0.0, 0.0, 0.08915899693965912), (0.0, 0.0, 0.0, 1.0), 0),
 (2, b'shoulder_lift_joint', 0, 8, 7, 1, 0.0, 0.0, -6.283185307179586, 6.283185307179586, 150.0, 3.15, b'upper_arm_link', (0.0, 1.0, 0.0), (0.0, 0.1358499974012375, 0.0), (0.0, -0.7071067966408575, 0.0, 0.7071067657322372), 1),
 (3, b'elbow_joint', 0, 9, 8, 1, 0.0, 0.0, -3.141592653589793, 3.141592653589793, 150.0, 3.15, b'forearm_link', (0.0, 1.0, 0.0), (0.0, -0.11969999969005585, 0.14500001072883606), (0.0, 0.0, 0.0, 1.0), 2),
 (4, b'wrist_1_joint', 0, 10, 9, 1, 0.0, 0.0, -6.283185307179586, 6.283185307179586, 28.0, 3.2, b'wrist_1_link', (0.0, 1.0, 0.0), (0.0, 0.0, 0.14225000143051147), (0.0, -0.7071067966408575, 0.0, 0.7071067657322372), 3),
 (5, b'wrist_2_joint', 0, 11, 10, 1, 0.0, 0.0, -6.283185307179586, 6.283185307179586, 28.0, 3.2, b'wrist_2_link', (0.0, 0.0, 1.0), (0.0, 0.09300000220537186, 0.0), (0.0, 0.0, 0.0, 1.0), 4),
 (6, b'wrist_3_joint', 0, 12, 11, 1, 0.0, 0.0, -6.283185307179586, 6.283185307179586, 28.0, 3.2, b'wrist_3_link', (0.0, 1.0, 0.0), (0.0, 0.0, 0.09465000033378601), (0.0, 0.0, 0.0, 1.0), 5),
 (7, b'ee_fixed_joint', 4, -1, -1, 0, 0.0, 0.0, 0.0, -1.0, 0.0, 0.0, b'ee_link', (0.0, 0.0, 0.0), (0.0, 0.08229999989271164, 0.0), (0.0, 0.0, -0.7071067966408575, 0.7071067657322372), 6),
 (8, b'wrist_3_link-tool0_fixed_joint', 4, -1, -1, 0, 0.0, 0.0, 0.0, -1.0, 0.0, 0.0, b'tool0', (0.0, 0.0, 0.0), (0.0, 0.08229999989271164, 0.0), (0.7071067966408575, 0.0, 0.0, 0.7071067657322372), 6),
 (9, b'base_link-base_fixed_joint', 4, -1, -1, 0, 0.0, 0.0, 0.0, -1.0, 0.0, 0.0, b'base', (0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.999999999999999, -4.371139000186241e-08), 0)]