Wall clock time in Isaac Gym benchmarks?

[ArthurAllshire]

NVIDIA Isaac Gym

Environment	PPO
Allegro Hand 1	3942.69
Ant	5466.3 +/- 279.61
Anymal	61.86 +/- 1.81
Anymal Terrain	19.82 +/- 0.57
Ball Balance	288.07 +/- 25.54
Cartpole 2	494.34 +/- 0.87
Franka Cabinet	3134.0 +/- 182.99
Humanoid	6474.34 +/- 696.27
Ingenuity	7066.82 +/- 488.97
Quadcopter	1237.75 +/- 127.05
Shadow Hand	7898.38 +/- 180.75

Environment	AMP
Humanoid	295.65 +/- 0.86

---

The following charts show the episode's mean length in timesteps (left) and the mean total reward (right)

Allegro Hand (PPO)

Ant (PPO)

Anymal (PPO)

Anymal Terrain (PPO)

Ball Balance (PPO)

Cartpole (PPO)

Franka Cabinet (PPO)

Humanoid (PPO)

Humanoid (AMP: imitate different pre-recorded human animations)

Ingenuity (PPO)

Quadcopter (PPO)

Shadow Hand (PPO)

Originally posted by @Toni-SM in #32 (comment)

Hi, I was looking through the benchmark results here (above) for Isaac Gym and was wondering if you could provide also wall clock time in them or if you have the info about how long it took to train each of them? Since for Isaac Gym training that is the critical variable for me to understand the performance due to the ability to vary the number of environments. Thanks :)

[Toni-SM]

Hi @ArthurAllshire

I still have to process the data... but in the following file you can find the time measurements of all the experiments shown in the charts. The experiments were performed on a Quadro RTX 6000 (If I remember well)

time.txt

The time is in seconds and it is measured for the whole script execution (env load + component initializations + learning)

For example:

# ppo_cartpole
for i in {1..10}
do
    start=`date +%s`
    python ppo_cartpole.py headless=True
    end=`date +%s`
    echo ppo_cartpole $((end-start)) >> time.txt
done

[Toni-SM]

Hi @ArthurAllshire

And, this is the time measurements for Omniverse Isaac Gym

time.txt

[ArthurAllshire]

Thanks, this is quite useful.

[ArthurAllshire]

By the way do you have plans to add recurrent training to this library?

[Toni-SM]

Yes. In fact, I'm working on it...