HumanoidVerse supports multiple simulators and tasks for humanoid robot sim-to-real learning. A key design logic is the separation and modularization of simulators, tasks, and algorithms, allowing for conviniently switching between simulators and tasks, and develop new ones with minimal efforts.

We compared the scope of HumanoidVerse with other sim-to-real frameworks and summarized the differences in supported features in the following table:

• Support for multiple simulators: (Currently) IsaacGym, Genesis, IsaacLab.
• Support for multiple embodiments: (Currently) Unitree Humanoid H1-10DoF, H1-19DoF, G1-12DoF, G1-23DoF.
• Sim-to-Sim and Sim-to-Real pipelines.
• Motion tracking tasks.

• 2025-02-04: 🎉 Initial Public Release! We have released the locomotion training pipeline for humanoid robots in IsaacGym, IsaacSim and Genesis.

Note:

• We recommend using mamba other than using conda, beacuse mamba is faster. Refer to Docs of Mamaba to install mamba.
• We are creating separate environments for each simulator, to avoid the dependency hell of different simulators.
• You do not need to install all of the simulators at once. If you only want to use IsaacGym, you can skip the installation of the other simulators.

git clone [email protected]:LeCAR-Lab/HumanoidVerse.git
cd HumanoidVerse

mamba create -n hgym python=3.8
mamba activate hgym

cd ../
wget https://developer.nvidia.com/isaac-gym-preview-4
tar -xvzf isaac-gym-preview-4

❯ tree . -L 1
.
├── isaacgym
└── HumanoidVerse

2 directories, 0 files

pip install -e ./isaacgym/python/.

cd isaacgym/python/examples/
python 1080_balls_of_solitude.py # or
python joint_monkey.py

# at the root of HumanoidVerse repository
pip install -e .

python humanoidverse/train_agent.py \
+simulator=isaacgym \
+exp=locomotion \
+domain_rand=NO_domain_rand \
+rewards=loco/reward_h1_locomotion \
+robot=h1/h1_10dof \
+terrain=terrain_locomotion_plane \
+obs=loco/leggedloco_obs_singlestep_withlinvel \
num_envs=1 \
project_name=TESTInstallation \
experiment_name=H110dof_loco_IsaacGym \
headless=False

git clone [email protected]:LeCAR-Lab/HumanoidVerse.git
cd HumanoidVerse

mamba create -n hsim python=3.10
mamba activate 

pip install -e .

python humanoidverse/train_agent.py \
+simulator=isaacsim \
+exp=locomotion \
+domain_rand=NO_domain_rand \
+rewards=loco/reward_h1_locomotion \
+robot=h1/h1_10dof \
+terrain=terrain_locomotion_plane \
+obs=loco/leggedloco_obs_singlestep_withlinvel \
num_envs=1 \
project_name=TESTInstallation \
experiment_name=H110dof_loco_IsaacSim \
headless=False

git clone [email protected]:LeCAR-Lab/HumanoidVerse.git
cd HumanoidVerse

mamba create -n hgen python=3.10
mamba activate hgen

pip install torch
pip install genesis-world==0.2.1

pip install -e .

python humanoidverse/train_agent.py \
+simulator=genesis \
+exp=locomotion \
+domain_rand=NO_domain_rand \
+rewards=loco/reward_h1_locomotion \
+robot=h1/h1_10dof \
+terrain=terrain_locomotion_plane \
+obs=loco/leggedloco_obs_singlestep_withlinvel \
num_envs=1 \
project_name=TESTInstallation \
experiment_name=H110dof_loco_Genesis \
headless=False

We support training & evluating in multiple simulators: IsaacGym, IsaacSim and Genesis by changing only ONE command line: +simulator=<simulator_name>

To train your policy, follow this command format:

python humanoidverse/train_agent.py \
+simulator=<simulator_name> \
+exp=<task_name> \
+domain_rand=<domain_randomization> \
+rewards=<reward_function> \
+robot=<robot_name> \
+terrain=<terrain_name> \
+obs=<observation_name> \
num_envs=<num_envs> \
project_name=<project_name> \
experiment_name=<experiment_name> \
headless=<headless_mode>

python humanoidverse/train_agent.py \
+simulator=isaacgym \
+exp=locomotion \
+domain_rand=NO_domain_rand \
+rewards=loco/reward_h1_locomotion \
+robot=h1/h1_10dof \
+terrain=terrain_locomotion_plane \
+obs=loco/leggedloco_obs_singlestep_withlinvel \
num_envs=4096 \
project_name=HumanoidLocomotion \
experiment_name=H110dof_loco_IsaacGym \
headless=True \
+opt=wandb

After running the training command, you can find the checkpoints and log files in the logs/<project_name>/<timestamp>-_<experiment_name>-<exp_type>-<robot_type> directory.

To evaluate the policy, follow this command format:

python humanoidverse/eval_agent.py +checkpoint=logs/xxx/../xx.pt

logs/xxx/../xx.pt is the relative path to the checkpoint file. You only need to run this command, our script will automatically find and load the training config.

python humanoidverse/eval_agent.py +checkpoint=logs/xxx/../xx.pt \
+domain_rand.push_robots=True \
+simulator=genesis # you can load the policy trained in isaacgym

Here are some starting commands to train & evaluate the locomotion policy on Unitree H1 Humanoid Robot among multiple simulators.

python humanoidverse/train_agent.py \
+simulator=isaacgym \
+exp=locomotion \
+domain_rand=NO_domain_rand \
+rewards=loco/reward_h1_locomotion \
+robot=h1/h1_10dof \
+terrain=terrain_locomotion_plane \
+obs=loco/leggedloco_obs_singlestep_withlinvel \
num_envs=4096 \
project_name=HumanoidLocomotion \
experiment_name=H110dof_loco_IsaacGym \
headless=True

After around 3000 epochs, evaluating in IsaacGym and Genesis:

python humanoidverse/train_agent.py \
+simulator=isaacsim \
+exp=locomotion \
+domain_rand=NO_domain_rand \
+rewards=loco/reward_h1_locomotion \
+robot=h1/h1 \
+terrain=terrain_locomotion_plane \
+obs=loco/leggedloco_obs_singlestep_withlinvel \
num_envs=4096 \
project_name=HumanoidLocomotion \
experiment_name=H119dof_loco_IsaacSim \
headless=True

After around 3000 epochs, evaluating in IsaacSim and Genesis:

python humanoidverse/train_agent.py \
+simulator=genesis \
+exp=locomotion \
+domain_rand=NO_domain_rand \
+rewards=loco/reward_h1_locomotion \
+robot=h1/h1_10dof \
+terrain=terrain_locomotion_plane \
+obs=loco/leggedloco_obs_singlestep_withlinvel \
num_envs=4096 \
project_name=HumanoidLocomotion \
experiment_name=H110dof_loco_Genesis \
headless=True \
rewards.reward_penalty_curriculum=True \
rewards.reward_initial_penalty_scale=0.5

After around 5000 epochs, evaluating in Genesis and IsaacGym:

This project is inspired by the following projects:

• ProtoMotions inspired us to use hydra for configuration management and influenced the overall structure of the codebase.
• Legged Gym provided the reference code for training locomotion tasks, handling domain randomizations, and designing reward functions. The starting point of our codebase is git clone [email protected]:leggedrobotics/legged_gym.git.
• RSL RL provided an example for the implementation of the PPO algorithm.

This project is made possible thanks to our amazing team members at LeCAR Lab:

• Gao Jiawei led the development of this project, designed the overall architecture, and implemented the core components, including the simulators, robots, tasks, and the training and evaluation framework.
• Tairan He implemented the design of domain randomizations, integrated the IsaacSim simulator (together with Zi Wang), and helped significantly with debugging in the early stages of the project.
• Wenli Xiao implemented the design of the observation dictionary, proprioception configuration, and history handlers, designed the actor-critic network architecture in PPO, and also helped greatly with debugging in the early stages of the project.
• Yuanhang Zhang contributed significantly to the sim-to-sim and sim-to-real pipelines and helped with debugging our PPO implementation.
• Zi Wang integrated the IsaacSim simulator into HumanoidVerse.
• Ziyan Xiong integrated the Genesis simulator into HumanoidVerse.
• Haotian Lin implemented MPPI in HumanoidVerse (to be released soon).
• Zeji Yi and Chaoyi Pan provided crucial help with our sim-to-sim pipeline in the early stages of the project.

Special thanks to Guanya Shi for his invaluable support and unwavering guidance throughout the project.

Please use the following bibtex if you find this repo helpful and would like to cite:

@misc{HumanoidVerse,
  author = {CMU LeCAR Lab},
  title = {HumanoidVerse: A Multi-Simulator Framework for Humanoid Robot Sim-to-Real Learning},
  year = {2025},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/LeCAR-Lab/HumanoidVerse}},
}

This project is licensed under the MIT License - see the LICENSE file for details.