Sim-to-Real Transfer Advice

[sesem738]

Hello Crazyswarm2 community,

I'm working on a project to perform sim2real transfer of quadcopter control tasks using the gym-pybullet-simulator. My goal is to train policies in simulation and then deploy them on real Crazyflie drones. I have access to a Vicon motion capture system for precise positioning data.
My current setup is as follows:

• Input: position, delta target position, orientation, linear velocity, angular velocity
• Output: PWM signals or attitude rates

I'm looking for advice and tips on how to effectively bridge the sim-to-real gap for this project using crazyswarm2:
What are the key differences between the gym-pybullet simulation and real Crazyflie dynamics that I should be aware of?

• Should I focus on outputting PWM signals or attitude rates for better sim-to-real performance?
• Are there any particular challenges or pitfalls I should watch out for when moving from simulation to real flights?
• For a simple circular trajectory task, after completing training in simulation, I'd like advice on transitioning from the drone's grounded state to a stable hover, and then executing the circular flight pattern. What are the recommended steps for this real-world implementation?

The plan is to start with a single agent and then scale up to multiple agents. Any insights or experiences you can share would be greatly appreciated. Once successful, I commit to publishing a comprehensive, step-by-step guide for Sim-to-Real with Crazyflies and Crazyswarm accompanied by videos. I believe this will serve as a valuable resource for the community.

[whoenig]

Both options have been explored, although perhaps not on the Crazyflie platform. Training a policy to output attitude rates and keeping an on-board attitude controller is easier (and thus perhaps the better starting point). One famous example is Davide Scaramuzzas work, where human pilots perform worse than an RL policy. It's also possible to directly learn motor forces, see https://sites.google.com/view/sim-to-multi-quad for an example (& discussion of difficulties). The same group has also extended this work to the multi-robot case with collision avoidance. In short, domain randomization seems to be sufficient to bridge the sim-to-real gap, although it remains an art to select what to randomize by how much.

For the circular trajectory task, the key idea is usually to have a "relative setpoint", so in simulation you always train to move to (0,0,0), but if you have a real trajectory, you change that relative error based on your actual goal.

[sesem738]

Hi @whoenig,

I've been trying to set up CrazySwarm2 with my Crazyflies but have encountered several issues:

1. Hello World Script Crash:

   • Using Vicon with mocap deck for Crazyflie
   • Edited crazyflie.yaml:
     • Set object name to match Vicon tracker (CF1)
     • Changed robot type to cf21_mocap_deck
   • Updated motion_capture.yaml with Vicon IP
   • Result: Drone flies up and crashes immediately
   • Warning: "No updated pose for cf1"
   • Note: libmotioncapture example provides pose data, and Bitcraze mocap example flies successfully

2. Xbox Controller Issue:

   • Attempted to run the Xbox controller example
   • Result: No response from the drone
   • Note: Can control drone in altitude assist mode with cfclient

3. Manual Flight Data Collection:

   • Planning to fly manually and collect pose data and attitute rates
   • Question: What's the best approach for this?

I'm happy to create separate issues for each problem if needed.

[whoenig]

1. Script crash:

• Without flying, check if the tf looks correct in rviz. If not, which tracking mode are you using? Basically you either have to use VICON tracker (and define rigid bodies there), or our software (and thus remove/disable rigid bodies in tracker, since we operate on the pointcloud).

2. Joystick

• Check the configuration (especially the remapping of the topics)
• Check that the joy topic works for you
• Check the topics that are outputted by the teleop node (eg. cf<x>/cmd_vel_legacy).

3. Data collection

• We always use the the usd-deck for this. It allows collecting data at a much higher rate and in a synchronized fashion than possible with the radio (e.g., you can log the IMU data at 500 Hz). Of course, you could also define your logtopics and then just use ros2 bag for data collection, if you don't care too much about frequency and latency.

[sesem738]

Hi @whoenig

1. I used the VICON tracker. I have been trying to debug it, but I still haven't had any success. I have recorded what the RVIZ looks like and what the Vicon Tracker sees as well. I have them available in this link for you to see. https://fltech-my.sharepoint.com/:f:/g/personal/knguyen_fit_edu/EqbFP0qSuoFDlKf38a73P30B8qT_9nNcVwAFQnlP9-iCww?e=WNwC6A.
   Whenever, I deactivate the mocap. I am able to run the hello word example, where it tries to hover and then lands, but since I guess it is only relying on the IMU the position is way off, but it works. However, when I enable the mocap. It immediately flips over and smashes in the wall. I tried the mocap example on the Bitcraze repository and when I set sendFullPose = True, it always crashes, but when I set it to False, it works. I am attaching my config filles so you can see if something is wrong in the setup.

2. I was able to control the crazyflie with my Xbox controller, but only when the mocap is not activated.

3. I have ordered the sd-deck as you recommended. Thanks for the suggestion

4. If I want to use the libobjecttracker, I see that I have to provide an initial position, is this the initial position from the vicon?
   config.zip

[whoenig]

1. I couldn't exactly see all the ROS messages in the video. Make sure your rigid body is defined such that the little antenna of the Crazyflie is positive x. In crazyflies.yaml, in the types, set motion_capture enabled to false (since you use Tracker for tracking). In Tracker, reduce the frequency of your cameras to 100 Hz.

2. It's just an initial guess, where the drone should roughly be taking of from. This is also used for the simulation. Note that there is no initial orientation, so the CF has to be facing forward (that is, the antenna pointing towards positive x of the world frame).