Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Avoid calling reset during env init #2770

Merged
merged 5 commits into from
Feb 10, 2025
Merged

[BugFix] Avoid calling reset during env init #2770

merged 5 commits into from
Feb 10, 2025

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Feb 7, 2025 •
edited
Loading

@vmoens
Update
f7bbbd3 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Feb 7, 2025
Merged
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 7, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2770

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Feb 7, 2025
@vmoens
[BugFix] Avoid calling reset during env init
8f747da 
ghstack-source-id: 080435dd10466498f083df6d2b74e9ef4b6d8640
Pull Request resolved: #2770
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 7, 2025
@vmoens vmoens mentioned this pull request Feb 7, 2025
Merged
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 7, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}21$. Worsened: $\large\color{#d91a1a}23$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.5597s 0.4630s 2.1599 Ops/s 2.1924 Ops/s $\color{#d91a1a}-1.48\%$
test_transformed 0.9224s 0.9137s 1.0944 Ops/s 1.0481 Ops/s $\color{#35bf28}+4.42\%$
test_serial 1.4045s 1.3899s 0.7195 Ops/s 0.7074 Ops/s $\color{#35bf28}+1.70\%$
test_parallel 1.2047s 1.1952s 0.8367 Ops/s 0.8112 Ops/s $\color{#35bf28}+3.13\%$
test_step_mdp_speed[True-True-True-True-True] 0.2684ms 29.9126μs 33.4308 KOps/s 31.6785 KOps/s $\textbf{\color{#35bf28}+5.53\%}$
test_step_mdp_speed[True-True-True-True-False] 79.6820μs 17.7501μs 56.3376 KOps/s 55.9855 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[True-True-True-False-True] 56.0650μs 17.0027μs 58.8143 KOps/s 58.1127 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[True-True-True-False-False] 53.1400μs 10.0272μs 99.7286 KOps/s 101.0623 KOps/s $\color{#d91a1a}-1.32\%$
test_step_mdp_speed[True-True-False-True-True] 80.5310μs 32.1819μs 31.0733 KOps/s 30.7607 KOps/s $\color{#35bf28}+1.02\%$
test_step_mdp_speed[True-True-False-True-False] 75.3010μs 19.7015μs 50.7575 KOps/s 50.8453 KOps/s $\color{#d91a1a}-0.17\%$
test_step_mdp_speed[True-True-False-False-True] 58.7700μs 18.7132μs 53.4383 KOps/s 52.1048 KOps/s $\color{#35bf28}+2.56\%$
test_step_mdp_speed[True-True-False-False-False] 51.8570μs 11.9127μs 83.9443 KOps/s 85.5160 KOps/s $\color{#d91a1a}-1.84\%$
test_step_mdp_speed[True-False-True-True-True] 86.0620μs 33.9252μs 29.4766 KOps/s 29.3607 KOps/s $\color{#35bf28}+0.39\%$
test_step_mdp_speed[True-False-True-True-False] 0.1996ms 21.4598μs 46.5987 KOps/s 46.2604 KOps/s $\color{#35bf28}+0.73\%$
test_step_mdp_speed[True-False-True-False-True] 0.1876ms 23.4210μs 42.6967 KOps/s 52.4225 KOps/s $\textbf{\color{#d91a1a}-18.55\%}$
test_step_mdp_speed[True-False-True-False-False] 37.9010μs 12.0143μs 83.2339 KOps/s 84.5691 KOps/s $\color{#d91a1a}-1.58\%$
test_step_mdp_speed[True-False-False-True-True] 74.6400μs 36.0011μs 27.7769 KOps/s 27.6986 KOps/s $\color{#35bf28}+0.28\%$
test_step_mdp_speed[True-False-False-True-False] 72.1560μs 23.4746μs 42.5992 KOps/s 42.8674 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[True-False-False-False-True] 51.2170μs 20.7863μs 48.1087 KOps/s 48.0970 KOps/s $\color{#35bf28}+0.02\%$
test_step_mdp_speed[True-False-False-False-False] 0.5128ms 13.9114μs 71.8834 KOps/s 74.2045 KOps/s $\color{#d91a1a}-3.13\%$
test_step_mdp_speed[False-True-True-True-True] 86.2320μs 33.9347μs 29.4683 KOps/s 29.3466 KOps/s $\color{#35bf28}+0.41\%$
test_step_mdp_speed[False-True-True-True-False] 59.7430μs 21.6012μs 46.2937 KOps/s 46.3458 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[False-True-True-False-True] 2.2950ms 21.5953μs 46.3063 KOps/s 45.5838 KOps/s $\color{#35bf28}+1.59\%$
test_step_mdp_speed[False-True-True-False-False] 50.2040μs 13.2484μs 75.4810 KOps/s 75.8343 KOps/s $\color{#d91a1a}-0.47\%$
test_step_mdp_speed[False-True-False-True-True] 0.1019ms 35.0300μs 28.5469 KOps/s 27.4879 KOps/s $\color{#35bf28}+3.85\%$
test_step_mdp_speed[False-True-False-True-False] 74.6100μs 23.1580μs 43.1816 KOps/s 42.5775 KOps/s $\color{#35bf28}+1.42\%$
test_step_mdp_speed[False-True-False-False-True] 94.7900μs 23.0048μs 43.4692 KOps/s 41.9781 KOps/s $\color{#35bf28}+3.55\%$
test_step_mdp_speed[False-True-False-False-False] 65.5910μs 14.9351μs 66.9562 KOps/s 66.6112 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[False-False-True-True-True] 97.0520μs 37.4234μs 26.7213 KOps/s 26.3506 KOps/s $\color{#35bf28}+1.41\%$
test_step_mdp_speed[False-False-True-True-False] 66.1440μs 26.5257μs 37.6992 KOps/s 39.6394 KOps/s $\color{#d91a1a}-4.89\%$
test_step_mdp_speed[False-False-True-False-True] 73.5480μs 23.1124μs 43.2669 KOps/s 42.2353 KOps/s $\color{#35bf28}+2.44\%$
test_step_mdp_speed[False-False-True-False-False] 38.9440μs 14.9858μs 66.7299 KOps/s 66.2744 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[False-False-False-True-True] 75.9730μs 38.8366μs 25.7489 KOps/s 25.0531 KOps/s $\color{#35bf28}+2.78\%$
test_step_mdp_speed[False-False-False-True-False] 99.8470μs 26.2644μs 38.0743 KOps/s 37.2760 KOps/s $\color{#35bf28}+2.14\%$
test_step_mdp_speed[False-False-False-False-True] 60.6540μs 24.7168μs 40.4583 KOps/s 39.2853 KOps/s $\color{#35bf28}+2.99\%$
test_step_mdp_speed[False-False-False-False-False] 91.5840μs 16.4422μs 60.8193 KOps/s 59.0706 KOps/s $\color{#35bf28}+2.96\%$
test_values[generalized_advantage_estimate-True-True] 10.8603ms 10.0100ms 99.9005 Ops/s 102.9875 Ops/s $\color{#d91a1a}-3.00\%$
test_values[vec_generalized_advantage_estimate-True-True] 26.2367ms 24.3116ms 41.1326 Ops/s 41.2079 Ops/s $\color{#d91a1a}-0.18\%$
test_values[td0_return_estimate-False-False] 0.2628ms 0.2199ms 4.5477 KOps/s 5.6657 KOps/s $\textbf{\color{#d91a1a}-19.73\%}$
test_values[td1_return_estimate-False-False] 25.7771ms 24.8215ms 40.2877 Ops/s 41.9583 Ops/s $\color{#d91a1a}-3.98\%$
test_values[vec_td1_return_estimate-False-False] 26.3687ms 24.5180ms 40.7863 Ops/s 41.5567 Ops/s $\color{#d91a1a}-1.85\%$
test_values[td_lambda_return_estimate-True-False] 35.8511ms 35.3637ms 28.2776 Ops/s 29.1944 Ops/s $\color{#d91a1a}-3.14\%$
test_values[vec_td_lambda_return_estimate-True-False] 26.3339ms 24.6312ms 40.5989 Ops/s 41.2865 Ops/s $\color{#d91a1a}-1.67\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.4958ms 8.5810ms 116.5369 Ops/s 119.8650 Ops/s $\color{#d91a1a}-2.78\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.5113ms 2.0140ms 496.5352 Ops/s 493.7577 Ops/s $\color{#35bf28}+0.56\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 1.5953ms 0.3682ms 2.7157 KOps/s 2.7532 KOps/s $\color{#d91a1a}-1.36\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.3342ms 44.4682ms 22.4880 Ops/s 22.2547 Ops/s $\color{#35bf28}+1.05\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.4145ms 3.5063ms 285.2011 Ops/s 285.5861 Ops/s $\color{#d91a1a}-0.13\%$
test_dqn_speed[False-None] 2.2444ms 1.3742ms 727.7024 Ops/s 705.8315 Ops/s $\color{#35bf28}+3.10\%$
test_dqn_speed[False-backward] 2.0655ms 1.8815ms 531.4919 Ops/s 532.4338 Ops/s $\color{#d91a1a}-0.18\%$
test_dqn_speed[True-None] 0.7859ms 0.4887ms 2.0461 KOps/s 2.0513 KOps/s $\color{#d91a1a}-0.26\%$
test_dqn_speed[True-backward] 1.0484ms 0.9096ms 1.0994 KOps/s 864.8286 Ops/s $\textbf{\color{#35bf28}+27.12\%}$
test_dqn_speed[reduce-overhead-None] 0.6327ms 0.4838ms 2.0670 KOps/s 2.0186 KOps/s $\color{#35bf28}+2.40\%$
test_dqn_speed[reduce-overhead-backward] 0.9415ms 0.8991ms 1.1122 KOps/s 1.0994 KOps/s $\color{#35bf28}+1.17\%$
test_ddpg_speed[False-None] 3.8442ms 2.8961ms 345.2951 Ops/s 332.5615 Ops/s $\color{#35bf28}+3.83\%$
test_ddpg_speed[False-backward] 4.9718ms 4.0473ms 247.0781 Ops/s 247.8627 Ops/s $\color{#d91a1a}-0.32\%$
test_ddpg_speed[True-None] 1.7955ms 1.2387ms 807.2884 Ops/s 812.0518 Ops/s $\color{#d91a1a}-0.59\%$
test_ddpg_speed[True-backward] 2.5030ms 2.2073ms 453.0370 Ops/s 464.6619 Ops/s $\color{#d91a1a}-2.50\%$
test_ddpg_speed[reduce-overhead-None] 1.6954ms 1.2500ms 799.9756 Ops/s 814.1239 Ops/s $\color{#d91a1a}-1.74\%$
test_ddpg_speed[reduce-overhead-backward] 2.9680ms 2.2465ms 445.1420 Ops/s 472.8397 Ops/s $\textbf{\color{#d91a1a}-5.86\%}$
test_sac_speed[False-None] 11.8337ms 8.4077ms 118.9380 Ops/s 123.8171 Ops/s $\color{#d91a1a}-3.94\%$
test_sac_speed[False-backward] 12.0501ms 11.1522ms 89.6683 Ops/s 92.7667 Ops/s $\color{#d91a1a}-3.34\%$
test_sac_speed[True-None] 2.8322ms 2.1804ms 458.6374 Ops/s 474.1639 Ops/s $\color{#d91a1a}-3.27\%$
test_sac_speed[True-backward] 8.4274ms 4.4754ms 223.4414 Ops/s 261.5742 Ops/s $\textbf{\color{#d91a1a}-14.58\%}$
test_sac_speed[reduce-overhead-None] 2.3115ms 2.0780ms 481.2256 Ops/s 464.7257 Ops/s $\color{#35bf28}+3.55\%$
test_sac_speed[reduce-overhead-backward] 3.8485ms 3.7509ms 266.6034 Ops/s 263.4913 Ops/s $\color{#35bf28}+1.18\%$
test_redq_speed[False-None] 15.3098ms 12.5431ms 79.7254 Ops/s 76.3299 Ops/s $\color{#35bf28}+4.45\%$
test_redq_speed[False-backward] 23.9083ms 21.6783ms 46.1290 Ops/s 43.2010 Ops/s $\textbf{\color{#35bf28}+6.78\%}$
test_redq_speed[True-None] 7.0468ms 4.7854ms 208.9707 Ops/s 199.7840 Ops/s $\color{#35bf28}+4.60\%$
test_redq_speed[True-backward] 13.5463ms 12.1367ms 82.3945 Ops/s 79.4296 Ops/s $\color{#35bf28}+3.73\%$
test_redq_speed[reduce-overhead-None] 6.0379ms 5.0489ms 198.0615 Ops/s 200.4640 Ops/s $\color{#d91a1a}-1.20\%$
test_redq_speed[reduce-overhead-backward] 14.0947ms 12.0857ms 82.7424 Ops/s 79.5279 Ops/s $\color{#35bf28}+4.04\%$
test_redq_deprec_speed[False-None] 14.2133ms 12.4973ms 80.0174 Ops/s 76.3576 Ops/s $\color{#35bf28}+4.79\%$
test_redq_deprec_speed[False-backward] 18.7143ms 18.0329ms 55.4541 Ops/s 53.3084 Ops/s $\color{#35bf28}+4.03\%$
test_redq_deprec_speed[True-None] 4.5651ms 3.7874ms 264.0363 Ops/s 257.7376 Ops/s $\color{#35bf28}+2.44\%$
test_redq_deprec_speed[True-backward] 9.1596ms 8.2930ms 120.5833 Ops/s 119.4732 Ops/s $\color{#35bf28}+0.93\%$
test_redq_deprec_speed[reduce-overhead-None] 4.5856ms 3.8163ms 262.0309 Ops/s 254.9217 Ops/s $\color{#35bf28}+2.79\%$
test_redq_deprec_speed[reduce-overhead-backward] 8.3064ms 8.1427ms 122.8091 Ops/s 120.5025 Ops/s $\color{#35bf28}+1.91\%$
test_td3_speed[False-None] 8.2676ms 7.8601ms 127.2247 Ops/s 122.4397 Ops/s $\color{#35bf28}+3.91\%$
test_td3_speed[False-backward] 11.2191ms 10.4080ms 96.0797 Ops/s 94.6782 Ops/s $\color{#35bf28}+1.48\%$
test_td3_speed[True-None] 1.8588ms 1.7664ms 566.1147 Ops/s 542.2399 Ops/s $\color{#35bf28}+4.40\%$
test_td3_speed[True-backward] 3.5152ms 3.3459ms 298.8700 Ops/s 281.6973 Ops/s $\textbf{\color{#35bf28}+6.10\%}$
test_td3_speed[reduce-overhead-None] 2.1059ms 1.7652ms 566.4944 Ops/s 538.4665 Ops/s $\textbf{\color{#35bf28}+5.21\%}$
test_td3_speed[reduce-overhead-backward] 3.9553ms 3.3655ms 297.1294 Ops/s 287.7919 Ops/s $\color{#35bf28}+3.24\%$
test_cql_speed[False-None] 39.4599ms 36.1817ms 27.6383 Ops/s 26.5865 Ops/s $\color{#35bf28}+3.96\%$
test_cql_speed[False-backward] 49.0722ms 45.6775ms 21.8926 Ops/s 20.5859 Ops/s $\textbf{\color{#35bf28}+6.35\%}$
test_cql_speed[True-None] 18.0039ms 16.0548ms 62.2865 Ops/s 63.0198 Ops/s $\color{#d91a1a}-1.16\%$
test_cql_speed[True-backward] 25.0191ms 22.6105ms 44.2273 Ops/s 44.9314 Ops/s $\color{#d91a1a}-1.57\%$
test_cql_speed[reduce-overhead-None] 18.1025ms 17.0173ms 58.7638 Ops/s 61.2921 Ops/s $\color{#d91a1a}-4.13\%$
test_cql_speed[reduce-overhead-backward] 25.3075ms 24.1832ms 41.3511 Ops/s 44.1907 Ops/s $\textbf{\color{#d91a1a}-6.43\%}$
test_a2c_speed[False-None] 10.7168ms 7.7337ms 129.3049 Ops/s 137.7866 Ops/s $\textbf{\color{#d91a1a}-6.16\%}$
test_a2c_speed[False-backward] 16.1398ms 15.3091ms 65.3207 Ops/s 69.4902 Ops/s $\textbf{\color{#d91a1a}-6.00\%}$
test_a2c_speed[True-None] 4.4222ms 4.0083ms 249.4797 Ops/s 267.8663 Ops/s $\textbf{\color{#d91a1a}-6.86\%}$
test_a2c_speed[True-backward] 11.6041ms 11.1953ms 89.3235 Ops/s 98.9157 Ops/s $\textbf{\color{#d91a1a}-9.70\%}$
test_a2c_speed[reduce-overhead-None] 5.4865ms 4.2161ms 237.1864 Ops/s 267.9597 Ops/s $\textbf{\color{#d91a1a}-11.48\%}$
test_a2c_speed[reduce-overhead-backward] 11.5992ms 11.3759ms 87.9055 Ops/s 98.4549 Ops/s $\textbf{\color{#d91a1a}-10.71\%}$
test_ppo_speed[False-None] 8.9145ms 7.9756ms 125.3829 Ops/s 132.8685 Ops/s $\textbf{\color{#d91a1a}-5.63\%}$
test_ppo_speed[False-backward] 16.1658ms 15.6639ms 63.8411 Ops/s 68.3337 Ops/s $\textbf{\color{#d91a1a}-6.57\%}$
test_ppo_speed[True-None] 5.7407ms 4.9392ms 202.4619 Ops/s 240.8461 Ops/s $\textbf{\color{#d91a1a}-15.94\%}$
test_ppo_speed[True-backward] 11.7215ms 11.0364ms 90.6096 Ops/s 99.5300 Ops/s $\textbf{\color{#d91a1a}-8.96\%}$
test_ppo_speed[reduce-overhead-None] 5.6872ms 4.6008ms 217.3543 Ops/s 241.8396 Ops/s $\textbf{\color{#d91a1a}-10.12\%}$
test_ppo_speed[reduce-overhead-backward] 10.8390ms 10.0457ms 99.5448 Ops/s 99.6862 Ops/s $\color{#d91a1a}-0.14\%$
test_reinforce_speed[False-None] 7.6811ms 6.7035ms 149.1768 Ops/s 149.7207 Ops/s $\color{#d91a1a}-0.36\%$
test_reinforce_speed[False-backward] 10.4912ms 9.8956ms 101.0550 Ops/s 98.7077 Ops/s $\color{#35bf28}+2.38\%$
test_reinforce_speed[True-None] 3.7351ms 3.1950ms 312.9869 Ops/s 324.6915 Ops/s $\color{#d91a1a}-3.60\%$
test_reinforce_speed[True-backward] 10.2726ms 9.1290ms 109.5416 Ops/s 111.8059 Ops/s $\color{#d91a1a}-2.03\%$
test_reinforce_speed[reduce-overhead-None] 3.4676ms 3.0671ms 326.0420 Ops/s 326.2600 Ops/s $\color{#d91a1a}-0.07\%$
test_reinforce_speed[reduce-overhead-backward] 11.0206ms 10.0212ms 99.7887 Ops/s 111.2579 Ops/s $\textbf{\color{#d91a1a}-10.31\%}$
test_iql_speed[False-None] 41.3490ms 34.2500ms 29.1971 Ops/s 31.3885 Ops/s $\textbf{\color{#d91a1a}-6.98\%}$
test_iql_speed[False-backward] 48.9169ms 46.1704ms 21.6589 Ops/s 22.0748 Ops/s $\color{#d91a1a}-1.88\%$
test_iql_speed[True-None] 12.5851ms 11.8054ms 84.7070 Ops/s 89.9820 Ops/s $\textbf{\color{#d91a1a}-5.86\%}$
test_iql_speed[True-backward] 24.1724ms 23.1925ms 43.1174 Ops/s 45.8035 Ops/s $\textbf{\color{#d91a1a}-5.86\%}$
test_iql_speed[reduce-overhead-None] 12.6556ms 11.6642ms 85.7325 Ops/s 84.4710 Ops/s $\color{#35bf28}+1.49\%$
test_iql_speed[reduce-overhead-backward] 24.8951ms 23.0318ms 43.4183 Ops/s 42.2014 Ops/s $\color{#35bf28}+2.88\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.4693ms 4.9064ms 203.8158 Ops/s 179.2254 Ops/s $\textbf{\color{#35bf28}+13.72\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7840ms 0.5151ms 1.9414 KOps/s 1.8329 KOps/s $\textbf{\color{#35bf28}+5.92\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7310ms 0.4905ms 2.0388 KOps/s 1.9476 KOps/s $\color{#35bf28}+4.68\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.8901ms 4.7307ms 211.3856 Ops/s 193.2136 Ops/s $\textbf{\color{#35bf28}+9.41\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.1297ms 0.5070ms 1.9723 KOps/s 1.8748 KOps/s $\textbf{\color{#35bf28}+5.20\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7896ms 0.4889ms 2.0456 KOps/s 1.9460 KOps/s $\textbf{\color{#35bf28}+5.12\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.8383ms 1.6462ms 607.4660 Ops/s 584.4787 Ops/s $\color{#35bf28}+3.93\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.2694ms 1.5585ms 641.6621 Ops/s 627.1890 Ops/s $\color{#35bf28}+2.31\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.5430ms 4.7781ms 209.2902 Ops/s 195.8656 Ops/s $\textbf{\color{#35bf28}+6.85\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.5174ms 0.6484ms 1.5422 KOps/s 1.4862 KOps/s $\color{#35bf28}+3.77\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0087ms 0.6215ms 1.6090 KOps/s 1.5226 KOps/s $\textbf{\color{#35bf28}+5.67\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.7767ms 4.5857ms 218.0715 Ops/s 204.0672 Ops/s $\textbf{\color{#35bf28}+6.86\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8689ms 0.5093ms 1.9634 KOps/s 1.9176 KOps/s $\color{#35bf28}+2.39\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8200ms 0.5069ms 1.9729 KOps/s 1.9491 KOps/s $\color{#35bf28}+1.22\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.8734ms 4.6419ms 215.4301 Ops/s 209.4705 Ops/s $\color{#35bf28}+2.85\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9805ms 0.4985ms 2.0059 KOps/s 1.8959 KOps/s $\textbf{\color{#35bf28}+5.80\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7734ms 0.4810ms 2.0790 KOps/s 2.0399 KOps/s $\color{#35bf28}+1.91\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.4958ms 4.7562ms 210.2504 Ops/s 198.3835 Ops/s $\textbf{\color{#35bf28}+5.98\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3283ms 0.6422ms 1.5571 KOps/s 1.4763 KOps/s $\textbf{\color{#35bf28}+5.48\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0313ms 0.6271ms 1.5946 KOps/s 1.5357 KOps/s $\color{#35bf28}+3.83\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 5.4481ms 4.1403ms 241.5277 Ops/s 226.6041 Ops/s $\textbf{\color{#35bf28}+6.59\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 7.1932ms 2.2649ms 441.5148 Ops/s 433.2603 Ops/s $\color{#35bf28}+1.91\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 4.9954ms 1.2929ms 773.4685 Ops/s 845.2458 Ops/s $\textbf{\color{#d91a1a}-8.49\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.4534s 13.2724ms 75.3443 Ops/s 232.1021 Ops/s $\textbf{\color{#d91a1a}-67.54\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 7.8525ms 2.2884ms 436.9896 Ops/s 429.5378 Ops/s $\color{#35bf28}+1.73\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.5712ms 1.3491ms 741.2479 Ops/s 778.5942 Ops/s $\color{#d91a1a}-4.80\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 5.7161ms 4.3195ms 231.5060 Ops/s 213.7441 Ops/s $\textbf{\color{#35bf28}+8.31\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.2418ms 2.5648ms 389.8982 Ops/s 412.2108 Ops/s $\textbf{\color{#d91a1a}-5.41\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.9069ms 1.4052ms 711.6609 Ops/s 642.3731 Ops/s $\textbf{\color{#35bf28}+10.79\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 12.0193ms 11.7024ms 85.4528 Ops/s 79.2641 Ops/s $\textbf{\color{#35bf28}+7.81\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 15.2234ms 14.0063ms 71.3966 Ops/s 70.5148 Ops/s $\color{#35bf28}+1.25\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 22.6898ms 20.9090ms 47.8264 Ops/s 46.6648 Ops/s $\color{#35bf28}+2.49\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 15.4054ms 14.2714ms 70.0704 Ops/s 68.3336 Ops/s $\color{#35bf28}+2.54\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 23.0287ms 20.9441ms 47.7461 Ops/s 47.2116 Ops/s $\color{#35bf28}+1.13\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 17.8361ms 15.7689ms 63.4160 Ops/s 63.2393 Ops/s $\color{#35bf28}+0.28\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 7, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}15$. Worsened: $\large\color{#d91a1a}11$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.8338s 0.7462s 1.3401 Ops/s 1.3275 Ops/s $\color{#35bf28}+0.95\%$
test_transformed 1.2978s 1.2971s 0.7710 Ops/s 0.7376 Ops/s $\color{#35bf28}+4.53\%$
test_serial 2.1490s 2.1440s 0.4664 Ops/s 0.4547 Ops/s $\color{#35bf28}+2.58\%$
test_parallel 1.8419s 1.7903s 0.5586 Ops/s 0.5527 Ops/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[True-True-True-True-True] 0.2536ms 41.1032μs 24.3290 KOps/s 24.7301 KOps/s $\color{#d91a1a}-1.62\%$
test_step_mdp_speed[True-True-True-True-False] 0.1527ms 23.6041μs 42.3656 KOps/s 42.8547 KOps/s $\color{#d91a1a}-1.14\%$
test_step_mdp_speed[True-True-True-False-True] 53.0910μs 22.6846μs 44.0828 KOps/s 45.1225 KOps/s $\color{#d91a1a}-2.30\%$
test_step_mdp_speed[True-True-True-False-False] 59.4010μs 13.1732μs 75.9118 KOps/s 77.6177 KOps/s $\color{#d91a1a}-2.20\%$
test_step_mdp_speed[True-True-False-True-True] 68.8610μs 43.4164μs 23.0328 KOps/s 23.8199 KOps/s $\color{#d91a1a}-3.30\%$
test_step_mdp_speed[True-True-False-True-False] 56.2110μs 25.6624μs 38.9676 KOps/s 39.8092 KOps/s $\color{#d91a1a}-2.11\%$
test_step_mdp_speed[True-True-False-False-True] 0.2083ms 24.9025μs 40.1566 KOps/s 41.0772 KOps/s $\color{#d91a1a}-2.24\%$
test_step_mdp_speed[True-True-False-False-False] 45.5010μs 15.5635μs 64.2531 KOps/s 65.4169 KOps/s $\color{#d91a1a}-1.78\%$
test_step_mdp_speed[True-False-True-True-True] 84.3220μs 45.7069μs 21.8785 KOps/s 22.6862 KOps/s $\color{#d91a1a}-3.56\%$
test_step_mdp_speed[True-False-True-True-False] 62.2010μs 28.2544μs 35.3928 KOps/s 35.9002 KOps/s $\color{#d91a1a}-1.41\%$
test_step_mdp_speed[True-False-True-False-True] 66.3710μs 25.2522μs 39.6005 KOps/s 41.6464 KOps/s $\color{#d91a1a}-4.91\%$
test_step_mdp_speed[True-False-True-False-False] 87.0810μs 15.7101μs 63.6535 KOps/s 66.5242 KOps/s $\color{#d91a1a}-4.32\%$
test_step_mdp_speed[True-False-False-True-True] 86.8310μs 47.8439μs 20.9013 KOps/s 21.8798 KOps/s $\color{#d91a1a}-4.47\%$
test_step_mdp_speed[True-False-False-True-False] 61.9510μs 29.9967μs 33.3370 KOps/s 33.7751 KOps/s $\color{#d91a1a}-1.30\%$
test_step_mdp_speed[True-False-False-False-True] 0.2240ms 27.2601μs 36.6837 KOps/s 38.7457 KOps/s $\textbf{\color{#d91a1a}-5.32\%}$
test_step_mdp_speed[True-False-False-False-False] 58.3510μs 17.8834μs 55.9179 KOps/s 59.5047 KOps/s $\textbf{\color{#d91a1a}-6.03\%}$
test_step_mdp_speed[False-True-True-True-True] 76.4210μs 45.5624μs 21.9479 KOps/s 22.9181 KOps/s $\color{#d91a1a}-4.23\%$
test_step_mdp_speed[False-True-True-True-False] 58.0010μs 28.2302μs 35.4231 KOps/s 36.3413 KOps/s $\color{#d91a1a}-2.53\%$
test_step_mdp_speed[False-True-True-False-True] 2.5689ms 29.4198μs 33.9907 KOps/s 35.5370 KOps/s $\color{#d91a1a}-4.35\%$
test_step_mdp_speed[False-True-True-False-False] 74.1210μs 17.3330μs 57.6934 KOps/s 59.5989 KOps/s $\color{#d91a1a}-3.20\%$
test_step_mdp_speed[False-True-False-True-True] 79.5610μs 47.5453μs 21.0326 KOps/s 21.5970 KOps/s $\color{#d91a1a}-2.61\%$
test_step_mdp_speed[False-True-False-True-False] 0.1084ms 30.5224μs 32.7628 KOps/s 33.5133 KOps/s $\color{#d91a1a}-2.24\%$
test_step_mdp_speed[False-True-False-False-True] 75.1010μs 31.0805μs 32.1745 KOps/s 33.1279 KOps/s $\color{#d91a1a}-2.88\%$
test_step_mdp_speed[False-True-False-False-False] 73.1710μs 19.4667μs 51.3699 KOps/s 52.1159 KOps/s $\color{#d91a1a}-1.43\%$
test_step_mdp_speed[False-False-True-True-True] 87.3510μs 49.7259μs 20.1103 KOps/s 20.6490 KOps/s $\color{#d91a1a}-2.61\%$
test_step_mdp_speed[False-False-True-True-False] 70.5310μs 32.5190μs 30.7513 KOps/s 30.8117 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[False-False-True-False-True] 72.3110μs 30.4492μs 32.8416 KOps/s 33.2493 KOps/s $\color{#d91a1a}-1.23\%$
test_step_mdp_speed[False-False-True-False-False] 58.9010μs 19.2968μs 51.8222 KOps/s 52.4049 KOps/s $\color{#d91a1a}-1.11\%$
test_step_mdp_speed[False-False-False-True-True] 90.6420μs 51.0650μs 19.5829 KOps/s 19.6887 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[False-False-False-True-False] 67.5710μs 35.0922μs 28.4964 KOps/s 28.8888 KOps/s $\color{#d91a1a}-1.36\%$
test_step_mdp_speed[False-False-False-False-True] 64.6410μs 32.6932μs 30.5874 KOps/s 31.3889 KOps/s $\color{#d91a1a}-2.55\%$
test_step_mdp_speed[False-False-False-False-False] 61.1400μs 21.4589μs 46.6006 KOps/s 47.1232 KOps/s $\color{#d91a1a}-1.11\%$
test_values[generalized_advantage_estimate-True-True] 25.5109ms 24.2888ms 41.1712 Ops/s 40.1475 Ops/s $\color{#35bf28}+2.55\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1098s 3.0937ms 323.2396 Ops/s 344.9870 Ops/s $\textbf{\color{#d91a1a}-6.30\%}$
test_values[td0_return_estimate-False-False] 0.1073ms 80.1385μs 12.4784 KOps/s 12.3970 KOps/s $\color{#35bf28}+0.66\%$
test_values[td1_return_estimate-False-False] 54.9468ms 54.3074ms 18.4137 Ops/s 18.1343 Ops/s $\color{#35bf28}+1.54\%$
test_values[vec_td1_return_estimate-False-False] 1.2944ms 1.0768ms 928.6935 Ops/s 916.2755 Ops/s $\color{#35bf28}+1.36\%$
test_values[td_lambda_return_estimate-True-False] 88.7853ms 86.7474ms 11.5277 Ops/s 11.3770 Ops/s $\color{#35bf28}+1.32\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.4327ms 1.0893ms 918.0380 Ops/s 921.8766 Ops/s $\color{#d91a1a}-0.42\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.6295ms 24.4422ms 40.9128 Ops/s 40.3963 Ops/s $\color{#35bf28}+1.28\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.0726ms 0.7514ms 1.3309 KOps/s 1.3231 KOps/s $\color{#35bf28}+0.59\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.9115ms 0.6702ms 1.4920 KOps/s 1.4839 KOps/s $\color{#35bf28}+0.54\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6277ms 1.4801ms 675.6163 Ops/s 675.0864 Ops/s $\color{#35bf28}+0.08\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8288ms 0.6840ms 1.4619 KOps/s 1.4527 KOps/s $\color{#35bf28}+0.63\%$
test_dqn_speed[False-None] 1.6640ms 1.5125ms 661.1656 Ops/s 651.0446 Ops/s $\color{#35bf28}+1.55\%$
test_dqn_speed[False-backward] 2.2308ms 2.1274ms 470.0674 Ops/s 464.0647 Ops/s $\color{#35bf28}+1.29\%$
test_dqn_speed[True-None] 0.6918ms 0.5505ms 1.8164 KOps/s 1.7622 KOps/s $\color{#35bf28}+3.08\%$
test_dqn_speed[True-backward] 1.2764ms 1.2182ms 820.9168 Ops/s 804.4973 Ops/s $\color{#35bf28}+2.04\%$
test_dqn_speed[reduce-overhead-None] 0.7842ms 0.5886ms 1.6989 KOps/s 1.7471 KOps/s $\color{#d91a1a}-2.76\%$
test_dqn_speed[reduce-overhead-backward] 1.1392ms 1.0702ms 934.4001 Ops/s 923.1633 Ops/s $\color{#35bf28}+1.22\%$
test_ddpg_speed[False-None] 3.3422ms 2.9102ms 343.6191 Ops/s 347.6555 Ops/s $\color{#d91a1a}-1.16\%$
test_ddpg_speed[False-backward] 4.4870ms 4.2482ms 235.3936 Ops/s 235.1045 Ops/s $\color{#35bf28}+0.12\%$
test_ddpg_speed[True-None] 1.4867ms 1.3406ms 745.9473 Ops/s 740.3047 Ops/s $\color{#35bf28}+0.76\%$
test_ddpg_speed[True-backward] 2.7599ms 2.5884ms 386.3424 Ops/s 384.5297 Ops/s $\color{#35bf28}+0.47\%$
test_ddpg_speed[reduce-overhead-None] 2.1244ms 1.3568ms 737.0440 Ops/s 735.4195 Ops/s $\color{#35bf28}+0.22\%$
test_ddpg_speed[reduce-overhead-backward] 2.1741ms 2.0413ms 489.8761 Ops/s 488.7664 Ops/s $\color{#35bf28}+0.23\%$
test_sac_speed[False-None] 8.6622ms 8.0833ms 123.7114 Ops/s 121.7937 Ops/s $\color{#35bf28}+1.57\%$
test_sac_speed[False-backward] 11.8504ms 11.3154ms 88.3750 Ops/s 87.8541 Ops/s $\color{#35bf28}+0.59\%$
test_sac_speed[True-None] 2.0084ms 1.8493ms 540.7380 Ops/s 538.5719 Ops/s $\color{#35bf28}+0.40\%$
test_sac_speed[True-backward] 3.6437ms 3.5698ms 280.1309 Ops/s 264.3338 Ops/s $\textbf{\color{#35bf28}+5.98\%}$
test_sac_speed[reduce-overhead-None] 21.6354ms 11.9896ms 83.4055 Ops/s 82.4486 Ops/s $\color{#35bf28}+1.16\%$
test_sac_speed[reduce-overhead-backward] 1.6977ms 1.6214ms 616.7514 Ops/s 548.8376 Ops/s $\textbf{\color{#35bf28}+12.37\%}$
test_redq_speed[False-None] 7.9230ms 7.5403ms 132.6215 Ops/s 131.1937 Ops/s $\color{#35bf28}+1.09\%$
test_redq_speed[False-backward] 11.5082ms 11.2447ms 88.9306 Ops/s 84.8155 Ops/s $\color{#35bf28}+4.85\%$
test_redq_speed[True-None] 2.4820ms 2.3269ms 429.7551 Ops/s 427.1977 Ops/s $\color{#35bf28}+0.60\%$
test_redq_speed[True-backward] 4.4551ms 3.9866ms 250.8376 Ops/s 246.5116 Ops/s $\color{#35bf28}+1.75\%$
test_redq_speed[reduce-overhead-None] 2.4990ms 2.3484ms 425.8132 Ops/s 423.4648 Ops/s $\color{#35bf28}+0.55\%$
test_redq_speed[reduce-overhead-backward] 4.4700ms 4.0045ms 249.7175 Ops/s 244.7900 Ops/s $\color{#35bf28}+2.01\%$
test_redq_deprec_speed[False-None] 9.6176ms 9.1391ms 109.4196 Ops/s 109.4240 Ops/s $-0.00\%$
test_redq_deprec_speed[False-backward] 12.5711ms 12.0272ms 83.1450 Ops/s 83.0060 Ops/s $\color{#35bf28}+0.17\%$
test_redq_deprec_speed[True-None] 2.8316ms 2.6482ms 377.6106 Ops/s 374.8347 Ops/s $\color{#35bf28}+0.74\%$
test_redq_deprec_speed[True-backward] 4.7401ms 4.3151ms 231.7422 Ops/s 227.1928 Ops/s $\color{#35bf28}+2.00\%$
test_redq_deprec_speed[reduce-overhead-None] 2.8428ms 2.6442ms 378.1888 Ops/s 373.3801 Ops/s $\color{#35bf28}+1.29\%$
test_redq_deprec_speed[reduce-overhead-backward] 4.3497ms 4.3021ms 232.4461 Ops/s 228.1354 Ops/s $\color{#35bf28}+1.89\%$
test_td3_speed[False-None] 7.9855ms 7.9516ms 125.7611 Ops/s 124.2860 Ops/s $\color{#35bf28}+1.19\%$
test_td3_speed[False-backward] 11.1776ms 10.2875ms 97.2057 Ops/s 96.7308 Ops/s $\color{#35bf28}+0.49\%$
test_td3_speed[True-None] 1.6770ms 1.6466ms 607.2987 Ops/s 595.4069 Ops/s $\color{#35bf28}+2.00\%$
test_td3_speed[True-backward] 3.2109ms 3.1794ms 314.5232 Ops/s 309.1414 Ops/s $\color{#35bf28}+1.74\%$
test_td3_speed[reduce-overhead-None] 50.9993ms 26.1988ms 38.1697 Ops/s 38.0104 Ops/s $\color{#35bf28}+0.42\%$
test_td3_speed[reduce-overhead-backward] 1.3771ms 1.3338ms 749.7471 Ops/s 732.3948 Ops/s $\color{#35bf28}+2.37\%$
test_cql_speed[False-None] 17.2119ms 16.8315ms 59.4124 Ops/s 59.1085 Ops/s $\color{#35bf28}+0.51\%$
test_cql_speed[False-backward] 22.5177ms 21.8635ms 45.7384 Ops/s 45.5390 Ops/s $\color{#35bf28}+0.44\%$
test_cql_speed[True-None] 3.4672ms 3.2787ms 304.9980 Ops/s 303.2309 Ops/s $\color{#35bf28}+0.58\%$
test_cql_speed[True-backward] 6.0014ms 5.4909ms 182.1210 Ops/s 175.3410 Ops/s $\color{#35bf28}+3.87\%$
test_cql_speed[reduce-overhead-None] 21.1955ms 13.1947ms 75.7879 Ops/s 75.8141 Ops/s $\color{#d91a1a}-0.03\%$
test_cql_speed[reduce-overhead-backward] 1.9787ms 1.8266ms 547.4561 Ops/s 506.9035 Ops/s $\textbf{\color{#35bf28}+8.00\%}$
test_a2c_speed[False-None] 3.3205ms 3.1910ms 313.3765 Ops/s 305.5707 Ops/s $\color{#35bf28}+2.55\%$
test_a2c_speed[False-backward] 7.1160ms 6.0842ms 164.3601 Ops/s 155.1485 Ops/s $\textbf{\color{#35bf28}+5.94\%}$
test_a2c_speed[True-None] 1.5093ms 1.3576ms 736.6200 Ops/s 736.8228 Ops/s $\color{#d91a1a}-0.03\%$
test_a2c_speed[True-backward] 3.0355ms 2.9014ms 344.6586 Ops/s 321.0321 Ops/s $\textbf{\color{#35bf28}+7.36\%}$
test_a2c_speed[reduce-overhead-None] 16.2126ms 9.1198ms 109.6515 Ops/s 112.0671 Ops/s $\color{#d91a1a}-2.16\%$
test_a2c_speed[reduce-overhead-backward] 1.5752ms 1.4668ms 681.7349 Ops/s 617.7782 Ops/s $\textbf{\color{#35bf28}+10.35\%}$
test_ppo_speed[False-None] 3.8774ms 3.7066ms 269.7885 Ops/s 265.8377 Ops/s $\color{#35bf28}+1.49\%$
test_ppo_speed[False-backward] 7.2154ms 6.7983ms 147.0952 Ops/s 140.1262 Ops/s $\color{#35bf28}+4.97\%$
test_ppo_speed[True-None] 1.6134ms 1.4289ms 699.8314 Ops/s 674.6404 Ops/s $\color{#35bf28}+3.73\%$
test_ppo_speed[True-backward] 3.1952ms 3.0611ms 326.6840 Ops/s 303.5353 Ops/s $\textbf{\color{#35bf28}+7.63\%}$
test_ppo_speed[reduce-overhead-None] 1.1085ms 0.9760ms 1.0245 KOps/s 1.0321 KOps/s $\color{#d91a1a}-0.73\%$
test_ppo_speed[reduce-overhead-backward] 1.5197ms 1.4215ms 703.4940 Ops/s 618.8544 Ops/s $\textbf{\color{#35bf28}+13.68\%}$
test_reinforce_speed[False-None] 2.4402ms 2.2929ms 436.1368 Ops/s 432.8486 Ops/s $\color{#35bf28}+0.76\%$
test_reinforce_speed[False-backward] 3.5281ms 3.3147ms 301.6833 Ops/s 291.7903 Ops/s $\color{#35bf28}+3.39\%$
test_reinforce_speed[True-None] 1.4718ms 1.3024ms 767.8364 Ops/s 758.1043 Ops/s $\color{#35bf28}+1.28\%$
test_reinforce_speed[True-backward] 3.0399ms 2.9468ms 339.3535 Ops/s 325.9758 Ops/s $\color{#35bf28}+4.10\%$
test_reinforce_speed[reduce-overhead-None] 19.7665ms 10.0073ms 99.9271 Ops/s 99.6201 Ops/s $\color{#35bf28}+0.31\%$
test_reinforce_speed[reduce-overhead-backward] 1.5742ms 1.5066ms 663.7255 Ops/s 595.1042 Ops/s $\textbf{\color{#35bf28}+11.53\%}$
test_iql_speed[False-None] 9.8880ms 9.3831ms 106.5748 Ops/s 106.2789 Ops/s $\color{#35bf28}+0.28\%$
test_iql_speed[False-backward] 13.7883ms 12.8998ms 77.5204 Ops/s 74.7565 Ops/s $\color{#35bf28}+3.70\%$
test_iql_speed[True-None] 2.5005ms 2.2349ms 447.4519 Ops/s 433.2988 Ops/s $\color{#35bf28}+3.27\%$
test_iql_speed[True-backward] 4.8680ms 4.7773ms 209.3212 Ops/s 203.4100 Ops/s $\color{#35bf28}+2.91\%$
test_iql_speed[reduce-overhead-None] 18.8051ms 11.1480ms 89.7025 Ops/s 90.0612 Ops/s $\color{#d91a1a}-0.40\%$
test_iql_speed[reduce-overhead-backward] 2.1096ms 1.9155ms 522.0556 Ops/s 500.2727 Ops/s $\color{#35bf28}+4.35\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.8988ms 6.3213ms 158.1962 Ops/s 156.0052 Ops/s $\color{#35bf28}+1.40\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5148ms 0.3075ms 3.2518 KOps/s 3.7103 KOps/s $\textbf{\color{#d91a1a}-12.36\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5867ms 0.3074ms 3.2529 KOps/s 4.0494 KOps/s $\textbf{\color{#d91a1a}-19.67\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.3066ms 5.9735ms 167.4063 Ops/s 163.6091 Ops/s $\color{#35bf28}+2.32\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.1314ms 0.2846ms 3.5132 KOps/s 3.4294 KOps/s $\color{#35bf28}+2.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5910ms 0.3116ms 3.2089 KOps/s 3.5123 KOps/s $\textbf{\color{#d91a1a}-8.64\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6538ms 1.4132ms 707.6021 Ops/s 770.0137 Ops/s $\textbf{\color{#d91a1a}-8.11\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5098ms 1.3051ms 766.1974 Ops/s 833.5839 Ops/s $\textbf{\color{#d91a1a}-8.08\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.3481ms 6.1709ms 162.0503 Ops/s 158.8025 Ops/s $\color{#35bf28}+2.05\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1165ms 0.4283ms 2.3346 KOps/s 2.1544 KOps/s $\textbf{\color{#35bf28}+8.36\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7639ms 0.4376ms 2.2850 KOps/s 2.5325 KOps/s $\textbf{\color{#d91a1a}-9.77\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.2954ms 6.1014ms 163.8958 Ops/s 162.8325 Ops/s $\color{#35bf28}+0.65\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5896ms 0.3302ms 3.0287 KOps/s 2.9763 KOps/s $\color{#35bf28}+1.76\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.4884ms 0.2598ms 3.8487 KOps/s 2.9834 KOps/s $\textbf{\color{#35bf28}+29.00\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.3521ms 6.0005ms 166.6529 Ops/s 162.2160 Ops/s $\color{#35bf28}+2.74\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.9648ms 0.2996ms 3.3380 KOps/s 3.2758 KOps/s $\color{#35bf28}+1.90\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5450ms 0.2941ms 3.4004 KOps/s 3.1632 KOps/s $\textbf{\color{#35bf28}+7.50\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.4180ms 6.2349ms 160.3878 Ops/s 159.1400 Ops/s $\color{#35bf28}+0.78\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0807ms 0.5012ms 1.9951 KOps/s 2.0300 KOps/s $\color{#d91a1a}-1.72\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7309ms 0.4791ms 2.0873 KOps/s 2.0924 KOps/s $\color{#d91a1a}-0.24\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 7.2468ms 5.6573ms 176.7618 Ops/s 180.3277 Ops/s $\color{#d91a1a}-1.98\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 6.1159ms 1.9882ms 502.9725 Ops/s 441.8429 Ops/s $\textbf{\color{#35bf28}+13.84\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 8.5815ms 1.2909ms 774.6734 Ops/s 776.1987 Ops/s $\color{#d91a1a}-0.20\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 8.7183ms 5.8304ms 171.5145 Ops/s 181.4258 Ops/s $\textbf{\color{#d91a1a}-5.46\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 8.7048ms 2.0407ms 490.0320 Ops/s 444.4887 Ops/s $\textbf{\color{#35bf28}+10.25\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.9471ms 1.2512ms 799.2023 Ops/s 801.1736 Ops/s $\color{#d91a1a}-0.25\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.5161s 16.1416ms 61.9518 Ops/s 31.1036 Ops/s $\textbf{\color{#35bf28}+99.18\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 10.1622ms 2.1630ms 462.3273 Ops/s 518.3092 Ops/s $\textbf{\color{#d91a1a}-10.80\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.2574ms 1.3965ms 716.0995 Ops/s 733.4461 Ops/s $\color{#d91a1a}-2.37\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 14.0651ms 13.6736ms 73.1334 Ops/s 72.4752 Ops/s $\color{#35bf28}+0.91\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 18.6491ms 17.0106ms 58.7869 Ops/s 59.1282 Ops/s $\color{#d91a1a}-0.58\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 18.2610ms 17.8512ms 56.0187 Ops/s 54.7301 Ops/s $\color{#35bf28}+2.35\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 18.3733ms 16.8708ms 59.2742 Ops/s 58.4121 Ops/s $\color{#35bf28}+1.48\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 18.0947ms 17.8445ms 56.0398 Ops/s 54.8479 Ops/s $\color{#35bf28}+2.17\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 19.5302ms 18.1713ms 55.0317 Ops/s 53.1709 Ops/s $\color{#35bf28}+3.50\%$

vmoens added 2 commits February 7, 2025 21:38
@vmoens
Update
90f86ab 
[ghstack-poisoned]
@vmoens
Update
3a42dad 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Feb 10, 2025
Merged
@vmoens
Update
3639256 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Feb 10, 2025
Merged
@vmoens vmoens added Suitable for minor Suitable to be integrated in minor release (no new feature) bug Something isn't working labels Feb 10, 2025
@vmoens
Update
0b79a28 
[ghstack-poisoned]
@vmoens vmoens merged commit 0b79a28 into gh/vmoens/87/base Feb 10, 2025
2 checks passed
vmoens added a commit that referenced this pull request Feb 10, 2025
@vmoens
[BugFix] Avoid calling reset during env init
09e93c1 
ghstack-source-id: 5ab8281c34aacfd7dbbfc0e285d88bcae0aededf
Pull Request resolved: #2770
@vmoens vmoens deleted the gh/vmoens/87/head branch February 10, 2025 12:27
vmoens added a commit that referenced this pull request Feb 10, 2025
@vmoens
[BugFix] Avoid calling reset during env init
28c3c7a 
ghstack-source-id: 5ab8281c34aacfd7dbbfc0e285d88bcae0aededf
Pull Request resolved: #2770

(cherry picked from commit 09e93c1)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Suitable for minor Suitable to be integrated in minor release (no new feature)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot