[Deprecation] Remove InPlaceSampler

[vmoens]

Stack from ghstack (oldest at bottom):

• [Deprecation] Gracing old *Spec with v0.8 versioning #2751
• -> [Deprecation] Remove InPlaceSampler #2750
• [Deprecation] Remove OrnsteinUhlenbeckProcessWrapper #2749
• [Deprecation] Remove AdditiveGaussianWrapper #2748
• [Deprecation] Remove NormalParamWrapper #2747
• [Deprecation] Change the default MLP depth #2746
• [CI] Upgrade to v0.7 #2745

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2750

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_simple	0.5258s	0.4449s	2.2478 Ops/s	2.2205 Ops/s	$\color{#35bf28}+1.23\%$
test_transformed	0.9896s	0.9080s	1.1013 Ops/s	1.0822 Ops/s	$\color{#35bf28}+1.77\%$
test_serial	1.4491s	1.3661s	0.7320 Ops/s	0.7225 Ops/s	$\color{#35bf28}+1.32\%$
test_parallel	1.2938s	1.2025s	0.8316 Ops/s	0.8131 Ops/s	$\color{#35bf28}+2.27\%$
test_step_mdp_speed[True-True-True-True-True]	0.3181ms	30.1756μs	33.1393 KOps/s	33.6438 KOps/s	$\color{#d91a1a}-1.50\%$
test_step_mdp_speed[True-True-True-True-False]	54.7820μs	18.4476μs	54.2076 KOps/s	57.7391 KOps/s	$\textbf{\color{#d91a1a}-6.12\%}$
test_step_mdp_speed[True-True-True-False-True]	49.7320μs	17.3512μs	57.6329 KOps/s	59.7812 KOps/s	$\color{#d91a1a}-3.59\%$
test_step_mdp_speed[True-True-True-False-False]	36.0970μs	10.2503μs	97.5584 KOps/s	102.5689 KOps/s	$\color{#d91a1a}-4.88\%$
test_step_mdp_speed[True-True-False-True-True]	76.0620μs	32.2364μs	31.0208 KOps/s	31.5472 KOps/s	$\color{#d91a1a}-1.67\%$
test_step_mdp_speed[True-True-False-True-False]	49.1320μs	20.0571μs	49.8577 KOps/s	52.3175 KOps/s	$\color{#d91a1a}-4.70\%$
test_step_mdp_speed[True-True-False-False-True]	59.0800μs	19.0827μs	52.4036 KOps/s	54.0218 KOps/s	$\color{#d91a1a}-3.00\%$
test_step_mdp_speed[True-True-False-False-False]	34.9150μs	12.1557μs	82.2663 KOps/s	85.2610 KOps/s	$\color{#d91a1a}-3.51\%$
test_step_mdp_speed[True-False-True-True-True]	72.1350μs	34.3561μs	29.1069 KOps/s	29.6133 KOps/s	$\color{#d91a1a}-1.71\%$
test_step_mdp_speed[True-False-True-True-False]	62.7170μs	21.8374μs	45.7929 KOps/s	47.3615 KOps/s	$\color{#d91a1a}-3.31\%$
test_step_mdp_speed[True-False-True-False-True]	45.6250μs	19.0802μs	52.4105 KOps/s	52.7452 KOps/s	$\color{#d91a1a}-0.63\%$
test_step_mdp_speed[True-False-True-False-False]	35.4460μs	12.2762μs	81.4584 KOps/s	85.6951 KOps/s	$\color{#d91a1a}-4.94\%$
test_step_mdp_speed[True-False-False-True-True]	87.0520μs	35.9517μs	27.8151 KOps/s	27.8994 KOps/s	$\color{#d91a1a}-0.30\%$
test_step_mdp_speed[True-False-False-True-False]	57.5070μs	23.8611μs	41.9092 KOps/s	43.6401 KOps/s	$\color{#d91a1a}-3.97\%$
test_step_mdp_speed[True-False-False-False-True]	48.9210μs	21.0502μs	47.5054 KOps/s	48.6473 KOps/s	$\color{#d91a1a}-2.35\%$
test_step_mdp_speed[True-False-False-False-False]	45.3250μs	14.1345μs	70.7487 KOps/s	74.9743 KOps/s	$\textbf{\color{#d91a1a}-5.64\%}$
test_step_mdp_speed[False-True-True-True-True]	78.5260μs	34.3376μs	29.1226 KOps/s	29.6969 KOps/s	$\color{#d91a1a}-1.93\%$
test_step_mdp_speed[False-True-True-True-False]	56.6550μs	22.1389μs	45.1694 KOps/s	47.2584 KOps/s	$\color{#d91a1a}-4.42\%$
test_step_mdp_speed[False-True-True-False-True]	68.1270μs	21.9383μs	45.5824 KOps/s	46.5279 KOps/s	$\color{#d91a1a}-2.03\%$
test_step_mdp_speed[False-True-True-False-False]	35.6760μs	13.7886μs	72.5235 KOps/s	76.8415 KOps/s	$\textbf{\color{#d91a1a}-5.62\%}$
test_step_mdp_speed[False-True-False-True-True]	84.2770μs	35.9444μs	27.8208 KOps/s	28.4006 KOps/s	$\color{#d91a1a}-2.04\%$
test_step_mdp_speed[False-True-False-True-False]	62.0660μs	24.0751μs	41.5366 KOps/s	43.6503 KOps/s	$\color{#d91a1a}-4.84\%$
test_step_mdp_speed[False-True-False-False-True]	2.7789ms	24.1383μs	41.4279 KOps/s	42.8807 KOps/s	$\color{#d91a1a}-3.39\%$
test_step_mdp_speed[False-True-False-False-False]	53.8800μs	15.5165μs	64.4474 KOps/s	68.0474 KOps/s	$\textbf{\color{#d91a1a}-5.29\%}$
test_step_mdp_speed[False-False-True-True-True]	82.3030μs	38.0865μs	26.2560 KOps/s	26.9894 KOps/s	$\color{#d91a1a}-2.72\%$
test_step_mdp_speed[False-False-True-True-False]	70.3610μs	25.8842μs	38.6336 KOps/s	40.1248 KOps/s	$\color{#d91a1a}-3.72\%$
test_step_mdp_speed[False-False-True-False-True]	54.1210μs	23.9223μs	41.8020 KOps/s	43.7021 KOps/s	$\color{#d91a1a}-4.35\%$
test_step_mdp_speed[False-False-True-False-False]	51.6400μs	15.5318μs	64.3842 KOps/s	67.5475 KOps/s	$\color{#d91a1a}-4.68\%$
test_step_mdp_speed[False-False-False-True-True]	81.2420μs	39.1767μs	25.5254 KOps/s	26.0833 KOps/s	$\color{#d91a1a}-2.14\%$
test_step_mdp_speed[False-False-False-True-False]	59.4810μs	27.1440μs	36.8405 KOps/s	38.1850 KOps/s	$\color{#d91a1a}-3.52\%$
test_step_mdp_speed[False-False-False-False-True]	54.2710μs	25.4491μs	39.2941 KOps/s	40.6661 KOps/s	$\color{#d91a1a}-3.37\%$
test_step_mdp_speed[False-False-False-False-False]	53.1390μs	17.1109μs	58.4422 KOps/s	60.9432 KOps/s	$\color{#d91a1a}-4.10\%$
test_values[generalized_advantage_estimate-True-True]	10.9582ms	9.5091ms	105.1628 Ops/s	104.7705 Ops/s	$\color{#35bf28}+0.37\%$
test_values[vec_generalized_advantage_estimate-True-True]	25.5025ms	23.9264ms	41.7949 Ops/s	41.3027 Ops/s	$\color{#35bf28}+1.19\%$
test_values[td0_return_estimate-False-False]	0.2301ms	0.1758ms	5.6870 KOps/s	5.4374 KOps/s	$\color{#35bf28}+4.59\%$
test_values[td1_return_estimate-False-False]	25.1874ms	23.6669ms	42.2531 Ops/s	41.8784 Ops/s	$\color{#35bf28}+0.89\%$
test_values[vec_td1_return_estimate-False-False]	26.5057ms	24.0101ms	41.6492 Ops/s	41.3375 Ops/s	$\color{#35bf28}+0.75\%$
test_values[td_lambda_return_estimate-True-False]	37.0329ms	34.1815ms	29.2556 Ops/s	28.9151 Ops/s	$\color{#35bf28}+1.18\%$
test_values[vec_td_lambda_return_estimate-True-False]	25.6035ms	24.0624ms	41.5586 Ops/s	41.5259 Ops/s	$\color{#35bf28}+0.08\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	9.4008ms	8.3846ms	119.2656 Ops/s	119.8799 Ops/s	$\color{#d91a1a}-0.51\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.3113ms	1.9093ms	523.7638 Ops/s	558.3742 Ops/s	$\textbf{\color{#d91a1a}-6.20\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4698ms	0.3639ms	2.7483 KOps/s	2.7528 KOps/s	$\color{#d91a1a}-0.16\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	53.4227ms	42.8158ms	23.3559 Ops/s	22.6767 Ops/s	$\color{#35bf28}+3.00\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	4.1271ms	3.4214ms	292.2745 Ops/s	268.8986 Ops/s	$\textbf{\color{#35bf28}+8.69\%}$
test_dqn_speed[False-None]	1.8829ms	1.4142ms	707.1313 Ops/s	713.0129 Ops/s	$\color{#d91a1a}-0.82\%$
test_dqn_speed[False-backward]	1.9460ms	1.8939ms	528.0205 Ops/s	529.0442 Ops/s	$\color{#d91a1a}-0.19\%$
test_dqn_speed[True-None]	0.7717ms	0.4817ms	2.0759 KOps/s	2.0846 KOps/s	$\color{#d91a1a}-0.42\%$
test_dqn_speed[True-backward]	0.9512ms	0.9009ms	1.1100 KOps/s	1.0932 KOps/s	$\color{#35bf28}+1.53\%$
test_dqn_speed[reduce-overhead-None]	0.5939ms	0.4831ms	2.0701 KOps/s	2.0619 KOps/s	$\color{#35bf28}+0.40\%$
test_dqn_speed[reduce-overhead-backward]	0.9619ms	0.9070ms	1.1025 KOps/s	1.0948 KOps/s	$\color{#35bf28}+0.71\%$
test_ddpg_speed[False-None]	3.6096ms	2.9015ms	344.6522 Ops/s	345.0588 Ops/s	$\color{#d91a1a}-0.12\%$
test_ddpg_speed[False-backward]	4.1519ms	4.0295ms	248.1688 Ops/s	248.6335 Ops/s	$\color{#d91a1a}-0.19\%$
test_ddpg_speed[True-None]	1.6158ms	1.2152ms	822.9244 Ops/s	814.1411 Ops/s	$\color{#35bf28}+1.08\%$
test_ddpg_speed[True-backward]	2.1543ms	2.1085ms	474.2798 Ops/s	471.5156 Ops/s	$\color{#35bf28}+0.59\%$
test_ddpg_speed[reduce-overhead-None]	1.4473ms	1.2111ms	825.7155 Ops/s	809.9717 Ops/s	$\color{#35bf28}+1.94\%$
test_ddpg_speed[reduce-overhead-backward]	2.1790ms	2.1038ms	475.3387 Ops/s	462.8703 Ops/s	$\color{#35bf28}+2.69\%$
test_sac_speed[False-None]	8.7728ms	8.0328ms	124.4895 Ops/s	124.2977 Ops/s	$\color{#35bf28}+0.15\%$
test_sac_speed[False-backward]	14.4262ms	10.9283ms	91.5059 Ops/s	93.5266 Ops/s	$\color{#d91a1a}-2.16\%$
test_sac_speed[True-None]	2.3095ms	2.0784ms	481.1302 Ops/s	480.5361 Ops/s	$\color{#35bf28}+0.12\%$
test_sac_speed[True-backward]	4.0083ms	3.7602ms	265.9448 Ops/s	267.3004 Ops/s	$\color{#d91a1a}-0.51\%$
test_sac_speed[reduce-overhead-None]	2.3327ms	2.0743ms	482.0809 Ops/s	477.2514 Ops/s	$\color{#35bf28}+1.01\%$
test_sac_speed[reduce-overhead-backward]	3.8604ms	3.7721ms	265.1012 Ops/s	264.6680 Ops/s	$\color{#35bf28}+0.16\%$
test_redq_speed[False-None]	14.5614ms	12.8976ms	77.5337 Ops/s	78.4664 Ops/s	$\color{#d91a1a}-1.19\%$
test_redq_speed[False-backward]	24.9071ms	22.0116ms	45.4305 Ops/s	45.0303 Ops/s	$\color{#35bf28}+0.89\%$
test_redq_speed[True-None]	5.5214ms	4.7595ms	210.1066 Ops/s	210.0555 Ops/s	$\color{#35bf28}+0.02\%$
test_redq_speed[True-backward]	13.1425ms	12.0536ms	82.9626 Ops/s	77.1248 Ops/s	$\textbf{\color{#35bf28}+7.57\%}$
test_redq_speed[reduce-overhead-None]	5.4870ms	4.7560ms	210.2613 Ops/s	208.6745 Ops/s	$\color{#35bf28}+0.76\%$
test_redq_speed[reduce-overhead-backward]	13.7057ms	12.1765ms	82.1253 Ops/s	73.7234 Ops/s	$\textbf{\color{#35bf28}+11.40\%}$
test_redq_deprec_speed[False-None]	14.5292ms	12.6658ms	78.9530 Ops/s	71.8220 Ops/s	$\textbf{\color{#35bf28}+9.93\%}$
test_redq_deprec_speed[False-backward]	19.7787ms	18.3162ms	54.5964 Ops/s	52.6270 Ops/s	$\color{#35bf28}+3.74\%$
test_redq_deprec_speed[True-None]	4.6630ms	3.8216ms	261.6724 Ops/s	262.2634 Ops/s	$\color{#d91a1a}-0.23\%$
test_redq_deprec_speed[True-backward]	8.2214ms	8.1296ms	123.0080 Ops/s	123.1432 Ops/s	$\color{#d91a1a}-0.11\%$
test_redq_deprec_speed[reduce-overhead-None]	4.5662ms	3.8056ms	262.7683 Ops/s	263.0501 Ops/s	$\color{#d91a1a}-0.11\%$
test_redq_deprec_speed[reduce-overhead-backward]	10.2221ms	8.2220ms	121.6246 Ops/s	122.6227 Ops/s	$\color{#d91a1a}-0.81\%$
test_td3_speed[False-None]	8.3810ms	8.0765ms	123.8164 Ops/s	125.5385 Ops/s	$\color{#d91a1a}-1.37\%$
test_td3_speed[False-backward]	10.9060ms	10.4812ms	95.4091 Ops/s	96.5826 Ops/s	$\color{#d91a1a}-1.22\%$
test_td3_speed[True-None]	1.8980ms	1.7766ms	562.8624 Ops/s	568.6106 Ops/s	$\color{#d91a1a}-1.01\%$
test_td3_speed[True-backward]	3.4373ms	3.3690ms	296.8237 Ops/s	300.1777 Ops/s	$\color{#d91a1a}-1.12\%$
test_td3_speed[reduce-overhead-None]	1.8954ms	1.7776ms	562.5688 Ops/s	566.9270 Ops/s	$\color{#d91a1a}-0.77\%$
test_td3_speed[reduce-overhead-backward]	4.4778ms	3.4088ms	293.3597 Ops/s	298.7833 Ops/s	$\color{#d91a1a}-1.82\%$
test_cql_speed[False-None]	41.9043ms	36.5963ms	27.3252 Ops/s	27.5302 Ops/s	$\color{#d91a1a}-0.74\%$
test_cql_speed[False-backward]	49.1023ms	46.4707ms	21.5189 Ops/s	21.7732 Ops/s	$\color{#d91a1a}-1.17\%$
test_cql_speed[True-None]	17.3893ms	15.7943ms	63.3139 Ops/s	60.3634 Ops/s	$\color{#35bf28}+4.89\%$
test_cql_speed[True-backward]	23.7533ms	22.3795ms	44.6838 Ops/s	42.9465 Ops/s	$\color{#35bf28}+4.05\%$
test_cql_speed[reduce-overhead-None]	18.1875ms	15.9259ms	62.7908 Ops/s	62.7551 Ops/s	$\color{#35bf28}+0.06\%$
test_cql_speed[reduce-overhead-backward]	24.1280ms	22.3270ms	44.7889 Ops/s	42.7051 Ops/s	$\color{#35bf28}+4.88\%$
test_a2c_speed[False-None]	8.3817ms	7.1491ms	139.8781 Ops/s	139.9689 Ops/s	$\color{#d91a1a}-0.06\%$
test_a2c_speed[False-backward]	16.3389ms	14.1346ms	70.7482 Ops/s	70.4970 Ops/s	$\color{#35bf28}+0.36\%$
test_a2c_speed[True-None]	4.1622ms	3.7070ms	269.7605 Ops/s	268.6601 Ops/s	$\color{#35bf28}+0.41\%$
test_a2c_speed[True-backward]	10.8028ms	10.1494ms	98.5277 Ops/s	98.7011 Ops/s	$\color{#d91a1a}-0.18\%$
test_a2c_speed[reduce-overhead-None]	4.5195ms	3.7055ms	269.8666 Ops/s	270.8206 Ops/s	$\color{#d91a1a}-0.35\%$
test_a2c_speed[reduce-overhead-backward]	11.3861ms	10.0975ms	99.0345 Ops/s	99.0133 Ops/s	$\color{#35bf28}+0.02\%$
test_ppo_speed[False-None]	9.1344ms	7.4888ms	133.5331 Ops/s	134.7351 Ops/s	$\color{#d91a1a}-0.89\%$
test_ppo_speed[False-backward]	16.2283ms	14.7113ms	67.9751 Ops/s	69.1516 Ops/s	$\color{#d91a1a}-1.70\%$
test_ppo_speed[True-None]	4.5184ms	4.0615ms	246.2153 Ops/s	245.9414 Ops/s	$\color{#35bf28}+0.11\%$
test_ppo_speed[True-backward]	10.2840ms	9.9106ms	100.9017 Ops/s	97.4254 Ops/s	$\color{#35bf28}+3.57\%$
test_ppo_speed[reduce-overhead-None]	5.2679ms	4.0596ms	246.3289 Ops/s	246.2181 Ops/s	$\color{#35bf28}+0.04\%$
test_ppo_speed[reduce-overhead-backward]	10.2251ms	9.8991ms	101.0189 Ops/s	100.0415 Ops/s	$\color{#35bf28}+0.98\%$
test_reinforce_speed[False-None]	7.9064ms	6.5379ms	152.9542 Ops/s	152.5234 Ops/s	$\color{#35bf28}+0.28\%$
test_reinforce_speed[False-backward]	11.0746ms	9.7950ms	102.0931 Ops/s	102.7644 Ops/s	$\color{#d91a1a}-0.65\%$
test_reinforce_speed[True-None]	3.7331ms	3.0268ms	330.3869 Ops/s	330.4177 Ops/s	$-0.01\%$
test_reinforce_speed[True-backward]	9.5228ms	8.9114ms	112.2162 Ops/s	111.6424 Ops/s	$\color{#35bf28}+0.51\%$
test_reinforce_speed[reduce-overhead-None]	4.5980ms	3.0606ms	326.7376 Ops/s	329.3474 Ops/s	$\color{#d91a1a}-0.79\%$
test_reinforce_speed[reduce-overhead-backward]	10.5585ms	8.9769ms	111.3967 Ops/s	111.8619 Ops/s	$\color{#d91a1a}-0.42\%$
test_iql_speed[False-None]	33.9279ms	32.2847ms	30.9744 Ops/s	30.7602 Ops/s	$\color{#35bf28}+0.70\%$
test_iql_speed[False-backward]	48.1998ms	45.3998ms	22.0266 Ops/s	21.8241 Ops/s	$\color{#35bf28}+0.93\%$
test_iql_speed[True-None]	12.8434ms	11.0087ms	90.8373 Ops/s	89.6088 Ops/s	$\color{#35bf28}+1.37\%$
test_iql_speed[True-backward]	22.7218ms	21.6616ms	46.1647 Ops/s	45.4902 Ops/s	$\color{#35bf28}+1.48\%$
test_iql_speed[reduce-overhead-None]	12.3270ms	11.0581ms	90.4312 Ops/s	90.1576 Ops/s	$\color{#35bf28}+0.30\%$
test_iql_speed[reduce-overhead-backward]	23.2130ms	21.7492ms	45.9786 Ops/s	45.8666 Ops/s	$\color{#35bf28}+0.24\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.8597ms	4.7497ms	210.5388 Ops/s	210.7649 Ops/s	$\color{#d91a1a}-0.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7725ms	0.5117ms	1.9543 KOps/s	1.9717 KOps/s	$\color{#d91a1a}-0.88\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.8851ms	0.4931ms	2.0282 KOps/s	2.0656 KOps/s	$\color{#d91a1a}-1.81\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	7.1707ms	4.5294ms	220.7797 Ops/s	223.0907 Ops/s	$\color{#d91a1a}-1.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.9656ms	0.5070ms	1.9725 KOps/s	1.9851 KOps/s	$\color{#d91a1a}-0.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6840ms	0.4801ms	2.0828 KOps/s	2.0955 KOps/s	$\color{#d91a1a}-0.61\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.7788ms	1.6590ms	602.7642 Ops/s	613.7574 Ops/s	$\color{#d91a1a}-1.79\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.3671ms	1.5759ms	634.5411 Ops/s	648.1877 Ops/s	$\color{#d91a1a}-2.11\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	7.5467ms	4.7241ms	211.6790 Ops/s	214.4924 Ops/s	$\color{#d91a1a}-1.31\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	2.4682ms	0.6538ms	1.5295 KOps/s	1.5627 KOps/s	$\color{#d91a1a}-2.13\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	1.0063ms	0.6305ms	1.5861 KOps/s	1.6145 KOps/s	$\color{#d91a1a}-1.76\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.7374ms	4.5650ms	219.0563 Ops/s	221.1462 Ops/s	$\color{#d91a1a}-0.95\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.1501ms	0.5226ms	1.9135 KOps/s	1.9863 KOps/s	$\color{#d91a1a}-3.67\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7017ms	0.4872ms	2.0525 KOps/s	2.0625 KOps/s	$\color{#d91a1a}-0.48\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.7076ms	4.5704ms	218.7978 Ops/s	225.6320 Ops/s	$\color{#d91a1a}-3.03\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.9339ms	0.5144ms	1.9439 KOps/s	1.9740 KOps/s	$\color{#d91a1a}-1.53\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7524ms	0.4834ms	2.0689 KOps/s	2.1123 KOps/s	$\color{#d91a1a}-2.06\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.2784ms	4.6969ms	212.9068 Ops/s	216.4427 Ops/s	$\color{#d91a1a}-1.63\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.2403ms	0.6598ms	1.5155 KOps/s	1.5625 KOps/s	$\color{#d91a1a}-3.00\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8399ms	0.6277ms	1.5930 KOps/s	1.6211 KOps/s	$\color{#d91a1a}-1.73\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	5.4864ms	4.2263ms	236.6144 Ops/s	248.3039 Ops/s	$\color{#d91a1a}-4.71\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	12.5645ms	2.7015ms	370.1663 Ops/s	448.6240 Ops/s	$\textbf{\color{#d91a1a}-17.49\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	4.4620ms	1.3321ms	750.7022 Ops/s	757.8383 Ops/s	$\color{#d91a1a}-0.94\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	5.5017ms	4.2514ms	235.2172 Ops/s	35.2488 Ops/s	$\textbf{\color{#35bf28}+567.31\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	7.5243ms	2.3337ms	428.4983 Ops/s	430.1436 Ops/s	$\color{#d91a1a}-0.38\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	3.8977ms	1.2823ms	779.8311 Ops/s	732.2506 Ops/s	$\textbf{\color{#35bf28}+6.50\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.4197s	12.6896ms	78.8049 Ops/s	226.1794 Ops/s	$\textbf{\color{#d91a1a}-65.16\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	4.9980ms	2.4467ms	408.7218 Ops/s	405.5216 Ops/s	$\color{#35bf28}+0.79\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	6.6825ms	1.5020ms	665.7790 Ops/s	634.2642 Ops/s	$\color{#35bf28}+4.97\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	11.9206ms	11.5550ms	86.5425 Ops/s	86.4556 Ops/s	$\color{#35bf28}+0.10\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	16.6675ms	14.5773ms	68.5998 Ops/s	70.0134 Ops/s	$\color{#d91a1a}-2.02\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	22.3713ms	20.3057ms	49.2473 Ops/s	49.0254 Ops/s	$\color{#35bf28}+0.45\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	15.5402ms	14.6388ms	68.3115 Ops/s	69.7859 Ops/s	$\color{#d91a1a}-2.11\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	22.7312ms	20.3505ms	49.1388 Ops/s	49.3997 Ops/s	$\color{#d91a1a}-0.53\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	17.2737ms	15.8485ms	63.0975 Ops/s	63.6753 Ops/s	$\color{#d91a1a}-0.91\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}19$. Worsened: $\large\color{#d91a1a}10$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_simple	0.8312s	0.7461s	1.3404 Ops/s	1.3709 Ops/s	$\color{#d91a1a}-2.23\%$
test_transformed	1.4287s	1.3435s	0.7443 Ops/s	0.7356 Ops/s	$\color{#35bf28}+1.18\%$
test_serial	2.1538s	2.1471s	0.4657 Ops/s	0.4510 Ops/s	$\color{#35bf28}+3.27\%$
test_parallel	1.8685s	1.8413s	0.5431 Ops/s	0.5464 Ops/s	$\color{#d91a1a}-0.61\%$
test_step_mdp_speed[True-True-True-True-True]	0.1413ms	38.7716μs	25.7920 KOps/s	25.3728 KOps/s	$\color{#35bf28}+1.65\%$
test_step_mdp_speed[True-True-True-True-False]	0.1555ms	23.2614μs	42.9897 KOps/s	41.9901 KOps/s	$\color{#35bf28}+2.38\%$
test_step_mdp_speed[True-True-True-False-True]	52.0410μs	21.8998μs	45.6626 KOps/s	43.5776 KOps/s	$\color{#35bf28}+4.78\%$
test_step_mdp_speed[True-True-True-False-False]	46.5900μs	12.9532μs	77.2009 KOps/s	74.5607 KOps/s	$\color{#35bf28}+3.54\%$
test_step_mdp_speed[True-True-False-True-True]	0.1036ms	42.4278μs	23.5695 KOps/s	23.1856 KOps/s	$\color{#35bf28}+1.66\%$
test_step_mdp_speed[True-True-False-True-False]	0.1462ms	25.5990μs	39.0640 KOps/s	37.9707 KOps/s	$\color{#35bf28}+2.88\%$
test_step_mdp_speed[True-True-False-False-True]	61.1210μs	25.1617μs	39.7430 KOps/s	39.4595 KOps/s	$\color{#35bf28}+0.72\%$
test_step_mdp_speed[True-True-False-False-False]	45.4710μs	15.3684μs	65.0684 KOps/s	63.1880 KOps/s	$\color{#35bf28}+2.98\%$
test_step_mdp_speed[True-False-True-True-True]	80.6210μs	44.8468μs	22.2982 KOps/s	21.7861 KOps/s	$\color{#35bf28}+2.35\%$
test_step_mdp_speed[True-False-True-True-False]	89.3820μs	27.0731μs	36.9370 KOps/s	35.3455 KOps/s	$\color{#35bf28}+4.50\%$
test_step_mdp_speed[True-False-True-False-True]	62.1110μs	24.1943μs	41.3321 KOps/s	39.1816 KOps/s	$\textbf{\color{#35bf28}+5.49\%}$
test_step_mdp_speed[True-False-True-False-False]	0.1924ms	15.4095μs	64.8952 KOps/s	63.8427 KOps/s	$\color{#35bf28}+1.65\%$
test_step_mdp_speed[True-False-False-True-True]	84.3210μs	46.2315μs	21.6303 KOps/s	20.8710 KOps/s	$\color{#35bf28}+3.64\%$
test_step_mdp_speed[True-False-False-True-False]	63.6910μs	29.7518μs	33.6114 KOps/s	32.3830 KOps/s	$\color{#35bf28}+3.79\%$
test_step_mdp_speed[True-False-False-False-True]	88.0010μs	26.4771μs	37.7685 KOps/s	36.6697 KOps/s	$\color{#35bf28}+3.00\%$
test_step_mdp_speed[True-False-False-False-False]	49.5810μs	17.4224μs	57.3974 KOps/s	55.0377 KOps/s	$\color{#35bf28}+4.29\%$
test_step_mdp_speed[False-True-True-True-True]	76.9010μs	45.1573μs	22.1448 KOps/s	21.6721 KOps/s	$\color{#35bf28}+2.18\%$
test_step_mdp_speed[False-True-True-True-False]	56.0210μs	27.7292μs	36.0630 KOps/s	34.8903 KOps/s	$\color{#35bf28}+3.36\%$
test_step_mdp_speed[False-True-True-False-True]	2.7801ms	28.6926μs	34.8522 KOps/s	33.8696 KOps/s	$\color{#35bf28}+2.90\%$
test_step_mdp_speed[False-True-True-False-False]	75.7410μs	17.2317μs	58.0327 KOps/s	57.7599 KOps/s	$\color{#35bf28}+0.47\%$
test_step_mdp_speed[False-True-False-True-True]	74.7410μs	46.5802μs	21.4683 KOps/s	20.9281 KOps/s	$\color{#35bf28}+2.58\%$
test_step_mdp_speed[False-True-False-True-False]	60.2610μs	30.0440μs	33.2845 KOps/s	32.1947 KOps/s	$\color{#35bf28}+3.39\%$
test_step_mdp_speed[False-True-False-False-True]	61.6610μs	30.9984μs	32.2597 KOps/s	31.1525 KOps/s	$\color{#35bf28}+3.55\%$
test_step_mdp_speed[False-True-False-False-False]	49.6910μs	19.3645μs	51.6410 KOps/s	50.6583 KOps/s	$\color{#35bf28}+1.94\%$
test_step_mdp_speed[False-False-True-True-True]	81.1710μs	49.3866μs	20.2484 KOps/s	20.0398 KOps/s	$\color{#35bf28}+1.04\%$
test_step_mdp_speed[False-False-True-True-False]	61.7110μs	32.6887μs	30.5917 KOps/s	30.2685 KOps/s	$\color{#35bf28}+1.07\%$
test_step_mdp_speed[False-False-True-False-True]	65.2020μs	30.6592μs	32.6167 KOps/s	32.4192 KOps/s	$\color{#35bf28}+0.61\%$
test_step_mdp_speed[False-False-True-False-False]	52.3710μs	19.3459μs	51.6906 KOps/s	50.3470 KOps/s	$\color{#35bf28}+2.67\%$
test_step_mdp_speed[False-False-False-True-True]	83.3420μs	50.9761μs	19.6170 KOps/s	19.1374 KOps/s	$\color{#35bf28}+2.51\%$
test_step_mdp_speed[False-False-False-True-False]	75.9910μs	34.8765μs	28.6726 KOps/s	28.1935 KOps/s	$\color{#35bf28}+1.70\%$
test_step_mdp_speed[False-False-False-False-True]	65.8410μs	32.5114μs	30.7585 KOps/s	30.6087 KOps/s	$\color{#35bf28}+0.49\%$
test_step_mdp_speed[False-False-False-False-False]	52.6110μs	21.5629μs	46.3759 KOps/s	45.3438 KOps/s	$\color{#35bf28}+2.28\%$
test_values[generalized_advantage_estimate-True-True]	25.9845ms	25.4158ms	39.3456 Ops/s	40.0708 Ops/s	$\color{#d91a1a}-1.81\%$
test_values[vec_generalized_advantage_estimate-True-True]	0.1233s	3.3653ms	297.1483 Ops/s	340.1236 Ops/s	$\textbf{\color{#d91a1a}-12.64\%}$
test_values[td0_return_estimate-False-False]	0.1182ms	80.0496μs	12.4923 KOps/s	12.5098 KOps/s	$\color{#d91a1a}-0.14\%$
test_values[td1_return_estimate-False-False]	60.8120ms	57.5656ms	17.3715 Ops/s	17.9746 Ops/s	$\color{#d91a1a}-3.36\%$
test_values[vec_td1_return_estimate-False-False]	1.3179ms	1.0870ms	919.9619 Ops/s	920.6143 Ops/s	$\color{#d91a1a}-0.07\%$
test_values[td_lambda_return_estimate-True-False]	92.7085ms	89.3672ms	11.1898 Ops/s	11.3949 Ops/s	$\color{#d91a1a}-1.80\%$
test_values[vec_td_lambda_return_estimate-True-False]	1.4373ms	1.0846ms	922.0109 Ops/s	920.3126 Ops/s	$\color{#35bf28}+0.18\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	25.5227ms	25.1772ms	39.7185 Ops/s	40.2016 Ops/s	$\color{#d91a1a}-1.20\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.0479ms	0.7570ms	1.3210 KOps/s	1.3294 KOps/s	$\color{#d91a1a}-0.63\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7644ms	0.6735ms	1.4847 KOps/s	1.4950 KOps/s	$\color{#d91a1a}-0.69\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.6429ms	1.4895ms	671.3485 Ops/s	671.5130 Ops/s	$\color{#d91a1a}-0.02\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.7278ms	0.6845ms	1.4608 KOps/s	1.4622 KOps/s	$\color{#d91a1a}-0.09\%$
test_dqn_speed[False-None]	1.6510ms	1.5235ms	656.3831 Ops/s	643.6192 Ops/s	$\color{#35bf28}+1.98\%$
test_dqn_speed[False-backward]	2.2749ms	2.1486ms	465.4149 Ops/s	461.5523 Ops/s	$\color{#35bf28}+0.84\%$
test_dqn_speed[True-None]	0.7126ms	0.5577ms	1.7932 KOps/s	1.7737 KOps/s	$\color{#35bf28}+1.10\%$
test_dqn_speed[True-backward]	1.2811ms	1.2224ms	818.0833 Ops/s	879.6875 Ops/s	$\textbf{\color{#d91a1a}-7.00\%}$
test_dqn_speed[reduce-overhead-None]	0.6287ms	0.5779ms	1.7303 KOps/s	1.7303 KOps/s	$+0.00\%$
test_dqn_speed[reduce-overhead-backward]	1.1407ms	1.0743ms	930.8113 Ops/s	1.0258 KOps/s	$\textbf{\color{#d91a1a}-9.26\%}$
test_ddpg_speed[False-None]	3.2411ms	2.8989ms	344.9600 Ops/s	340.8697 Ops/s	$\color{#35bf28}+1.20\%$
test_ddpg_speed[False-backward]	4.8243ms	4.3165ms	231.6718 Ops/s	235.6586 Ops/s	$\color{#d91a1a}-1.69\%$
test_ddpg_speed[True-None]	1.5082ms	1.3370ms	747.9249 Ops/s	740.1193 Ops/s	$\color{#35bf28}+1.05\%$
test_ddpg_speed[True-backward]	2.7033ms	2.5640ms	390.0170 Ops/s	406.7406 Ops/s	$\color{#d91a1a}-4.11\%$
test_ddpg_speed[reduce-overhead-None]	1.5378ms	1.3522ms	739.5549 Ops/s	729.0035 Ops/s	$\color{#35bf28}+1.45\%$
test_ddpg_speed[reduce-overhead-backward]	2.1919ms	2.0330ms	491.8761 Ops/s	523.0017 Ops/s	$\textbf{\color{#d91a1a}-5.95\%}$
test_sac_speed[False-None]	8.5541ms	8.1136ms	123.2496 Ops/s	122.5031 Ops/s	$\color{#35bf28}+0.61\%$
test_sac_speed[False-backward]	11.7224ms	11.2602ms	88.8082 Ops/s	90.0474 Ops/s	$\color{#d91a1a}-1.38\%$
test_sac_speed[True-None]	1.8939ms	1.8341ms	545.2293 Ops/s	533.3529 Ops/s	$\color{#35bf28}+2.23\%$
test_sac_speed[True-backward]	4.1719ms	3.7271ms	268.3029 Ops/s	264.3563 Ops/s	$\color{#35bf28}+1.49\%$
test_sac_speed[reduce-overhead-None]	21.7402ms	12.1809ms	82.0954 Ops/s	83.4658 Ops/s	$\color{#d91a1a}-1.64\%$
test_sac_speed[reduce-overhead-backward]	1.9147ms	1.7819ms	561.2067 Ops/s	547.5443 Ops/s	$\color{#35bf28}+2.50\%$
test_redq_speed[False-None]	8.0093ms	7.5761ms	131.9937 Ops/s	129.8334 Ops/s	$\color{#35bf28}+1.66\%$
test_redq_speed[False-backward]	14.0622ms	11.8221ms	84.5870 Ops/s	84.0540 Ops/s	$\color{#35bf28}+0.63\%$
test_redq_speed[True-None]	2.4691ms	2.2963ms	435.4913 Ops/s	426.4934 Ops/s	$\color{#35bf28}+2.11\%$
test_redq_speed[True-backward]	4.6410ms	4.1805ms	239.2053 Ops/s	234.8043 Ops/s	$\color{#35bf28}+1.87\%$
test_redq_speed[reduce-overhead-None]	2.5084ms	2.3061ms	433.6372 Ops/s	422.9965 Ops/s	$\color{#35bf28}+2.52\%$
test_redq_speed[reduce-overhead-backward]	4.3786ms	4.1716ms	239.7186 Ops/s	235.2199 Ops/s	$\color{#35bf28}+1.91\%$
test_redq_deprec_speed[False-None]	9.4643ms	9.1310ms	109.5172 Ops/s	107.4905 Ops/s	$\color{#35bf28}+1.89\%$
test_redq_deprec_speed[False-backward]	12.7833ms	12.3587ms	80.9149 Ops/s	80.1035 Ops/s	$\color{#35bf28}+1.01\%$
test_redq_deprec_speed[True-None]	2.8641ms	2.6435ms	378.2920 Ops/s	373.5998 Ops/s	$\color{#35bf28}+1.26\%$
test_redq_deprec_speed[True-backward]	4.8974ms	4.4710ms	223.6653 Ops/s	219.2462 Ops/s	$\color{#35bf28}+2.02\%$
test_redq_deprec_speed[reduce-overhead-None]	2.7878ms	2.6263ms	380.7692 Ops/s	374.0906 Ops/s	$\color{#35bf28}+1.79\%$
test_redq_deprec_speed[reduce-overhead-backward]	4.8330ms	4.4807ms	223.1818 Ops/s	220.1277 Ops/s	$\color{#35bf28}+1.39\%$
test_td3_speed[False-None]	8.3309ms	8.0688ms	123.9344 Ops/s	122.4132 Ops/s	$\color{#35bf28}+1.24\%$
test_td3_speed[False-backward]	11.4353ms	10.6672ms	93.7451 Ops/s	93.9141 Ops/s	$\color{#d91a1a}-0.18\%$
test_td3_speed[True-None]	1.6573ms	1.6336ms	612.1394 Ops/s	587.3199 Ops/s	$\color{#35bf28}+4.23\%$
test_td3_speed[True-backward]	3.3688ms	3.3004ms	302.9978 Ops/s	289.5882 Ops/s	$\color{#35bf28}+4.63\%$
test_td3_speed[reduce-overhead-None]	55.1400ms	26.6498ms	37.5238 Ops/s	36.2530 Ops/s	$\color{#35bf28}+3.51\%$
test_td3_speed[reduce-overhead-backward]	1.5321ms	1.4960ms	668.4661 Ops/s	649.5188 Ops/s	$\color{#35bf28}+2.92\%$
test_cql_speed[False-None]	17.8690ms	17.0050ms	58.8061 Ops/s	58.7341 Ops/s	$\color{#35bf28}+0.12\%$
test_cql_speed[False-backward]	23.3656ms	22.4595ms	44.5245 Ops/s	44.3152 Ops/s	$\color{#35bf28}+0.47\%$
test_cql_speed[True-None]	3.3397ms	3.2511ms	307.5877 Ops/s	301.0369 Ops/s	$\color{#35bf28}+2.18\%$
test_cql_speed[True-backward]	6.3631ms	5.7197ms	174.8352 Ops/s	175.0615 Ops/s	$\color{#d91a1a}-0.13\%$
test_cql_speed[reduce-overhead-None]	20.8947ms	13.1525ms	76.0312 Ops/s	57.4557 Ops/s	$\textbf{\color{#35bf28}+32.33\%}$
test_cql_speed[reduce-overhead-backward]	2.1394ms	1.9994ms	500.1592 Ops/s	492.2284 Ops/s	$\color{#35bf28}+1.61\%$
test_a2c_speed[False-None]	3.4242ms	3.2245ms	310.1272 Ops/s	305.9249 Ops/s	$\color{#35bf28}+1.37\%$
test_a2c_speed[False-backward]	6.9774ms	6.3641ms	157.1324 Ops/s	155.6942 Ops/s	$\color{#35bf28}+0.92\%$
test_a2c_speed[True-None]	1.5322ms	1.3474ms	742.1934 Ops/s	735.6257 Ops/s	$\color{#35bf28}+0.89\%$
test_a2c_speed[True-backward]	3.1052ms	2.8730ms	348.0694 Ops/s	320.2346 Ops/s	$\textbf{\color{#35bf28}+8.69\%}$
test_a2c_speed[reduce-overhead-None]	15.8605ms	9.0666ms	110.2945 Ops/s	110.3089 Ops/s	$\color{#d91a1a}-0.01\%$
test_a2c_speed[reduce-overhead-backward]	1.8586ms	1.4698ms	680.3754 Ops/s	611.6612 Ops/s	$\textbf{\color{#35bf28}+11.23\%}$
test_ppo_speed[False-None]	4.0811ms	3.7129ms	269.3296 Ops/s	264.4126 Ops/s	$\color{#35bf28}+1.86\%$
test_ppo_speed[False-backward]	7.2737ms	6.8765ms	145.4221 Ops/s	139.7251 Ops/s	$\color{#35bf28}+4.08\%$
test_ppo_speed[True-None]	1.8148ms	1.4066ms	710.9185 Ops/s	697.2220 Ops/s	$\color{#35bf28}+1.96\%$
test_ppo_speed[True-backward]	3.1144ms	3.0512ms	327.7411 Ops/s	304.0436 Ops/s	$\textbf{\color{#35bf28}+7.79\%}$
test_ppo_speed[reduce-overhead-None]	1.3692ms	0.9761ms	1.0245 KOps/s	1.0199 KOps/s	$\color{#35bf28}+0.45\%$
test_ppo_speed[reduce-overhead-backward]	1.5531ms	1.4067ms	710.8827 Ops/s	616.5578 Ops/s	$\textbf{\color{#35bf28}+15.30\%}$
test_reinforce_speed[False-None]	2.7022ms	2.2969ms	435.3613 Ops/s	427.0719 Ops/s	$\color{#35bf28}+1.94\%$
test_reinforce_speed[False-backward]	3.9589ms	3.3125ms	301.8909 Ops/s	288.1346 Ops/s	$\color{#35bf28}+4.77\%$
test_reinforce_speed[True-None]	1.6901ms	1.2942ms	772.6949 Ops/s	757.5530 Ops/s	$\color{#35bf28}+2.00\%$
test_reinforce_speed[True-backward]	3.0054ms	2.9179ms	342.7125 Ops/s	324.0763 Ops/s	$\textbf{\color{#35bf28}+5.75\%}$
test_reinforce_speed[reduce-overhead-None]	18.4122ms	10.1628ms	98.3982 Ops/s	99.9229 Ops/s	$\color{#d91a1a}-1.53\%$
test_reinforce_speed[reduce-overhead-backward]	1.6072ms	1.5656ms	638.7328 Ops/s	599.2375 Ops/s	$\textbf{\color{#35bf28}+6.59\%}$
test_iql_speed[False-None]	9.8203ms	9.2635ms	107.9503 Ops/s	104.8946 Ops/s	$\color{#35bf28}+2.91\%$
test_iql_speed[False-backward]	13.3675ms	13.0265ms	76.7663 Ops/s	74.4314 Ops/s	$\color{#35bf28}+3.14\%$
test_iql_speed[True-None]	2.4215ms	2.2298ms	448.4744 Ops/s	431.0275 Ops/s	$\color{#35bf28}+4.05\%$
test_iql_speed[True-backward]	5.3875ms	4.9072ms	203.7806 Ops/s	199.1668 Ops/s	$\color{#35bf28}+2.32\%$
test_iql_speed[reduce-overhead-None]	18.9046ms	11.2165ms	89.1545 Ops/s	88.7629 Ops/s	$\color{#35bf28}+0.44\%$
test_iql_speed[reduce-overhead-backward]	2.2399ms	2.0626ms	484.8269 Ops/s	461.7544 Ops/s	$\color{#35bf28}+5.00\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	8.0883ms	6.3496ms	157.4904 Ops/s	154.0120 Ops/s	$\color{#35bf28}+2.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6449ms	0.3755ms	2.6633 KOps/s	3.1630 KOps/s	$\textbf{\color{#d91a1a}-15.80\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7552ms	0.3575ms	2.7974 KOps/s	3.2896 KOps/s	$\textbf{\color{#d91a1a}-14.96\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.3995ms	6.0934ms	164.1121 Ops/s	163.4174 Ops/s	$\color{#35bf28}+0.43\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.0343ms	0.3159ms	3.1653 KOps/s	3.3845 KOps/s	$\textbf{\color{#d91a1a}-6.47\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6896ms	0.2829ms	3.5352 KOps/s	3.3488 KOps/s	$\textbf{\color{#35bf28}+5.57\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.5538ms	1.3222ms	756.3240 Ops/s	715.8734 Ops/s	$\textbf{\color{#35bf28}+5.65\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.5375ms	1.2746ms	784.5492 Ops/s	775.8732 Ops/s	$\color{#35bf28}+1.12\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.4745ms	6.2850ms	159.1086 Ops/s	157.0040 Ops/s	$\color{#35bf28}+1.34\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.9400ms	0.4148ms	2.4109 KOps/s	2.4059 KOps/s	$\color{#35bf28}+0.21\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6400ms	0.3942ms	2.5370 KOps/s	2.4882 KOps/s	$\color{#35bf28}+1.96\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.3280ms	6.1211ms	163.3706 Ops/s	163.5383 Ops/s	$\color{#d91a1a}-0.10\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.8919ms	0.2781ms	3.5963 KOps/s	2.9564 KOps/s	$\textbf{\color{#35bf28}+21.65\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5128ms	0.2791ms	3.5827 KOps/s	3.1898 KOps/s	$\textbf{\color{#35bf28}+12.32\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	10.0926ms	6.1499ms	162.6037 Ops/s	163.3647 Ops/s	$\color{#d91a1a}-0.47\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7086ms	0.3136ms	3.1890 KOps/s	2.7459 KOps/s	$\textbf{\color{#35bf28}+16.14\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5356ms	0.2859ms	3.4980 KOps/s	2.8350 KOps/s	$\textbf{\color{#35bf28}+23.39\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.4839ms	6.2329ms	160.4382 Ops/s	157.0478 Ops/s	$\color{#35bf28}+2.16\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1771ms	0.4133ms	2.4197 KOps/s	2.2222 KOps/s	$\textbf{\color{#35bf28}+8.89\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6597ms	0.4402ms	2.2718 KOps/s	2.2898 KOps/s	$\color{#d91a1a}-0.79\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	7.1791ms	5.5064ms	181.6070 Ops/s	177.4532 Ops/s	$\color{#35bf28}+2.34\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	6.5606ms	2.1166ms	472.4570 Ops/s	425.2128 Ops/s	$\textbf{\color{#35bf28}+11.11\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	9.2210ms	1.3141ms	761.0036 Ops/s	827.8329 Ops/s	$\textbf{\color{#d91a1a}-8.07\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	8.4028ms	5.6847ms	175.9102 Ops/s	181.3557 Ops/s	$\color{#d91a1a}-3.00\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	7.7560ms	2.1115ms	473.6027 Ops/s	450.1619 Ops/s	$\textbf{\color{#35bf28}+5.21\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	8.2718ms	1.3325ms	750.4821 Ops/s	771.0656 Ops/s	$\color{#d91a1a}-2.67\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.5022s	15.7751ms	63.3911 Ops/s	31.2406 Ops/s	$\textbf{\color{#35bf28}+102.91\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	10.0089ms	2.3159ms	431.8058 Ops/s	463.9687 Ops/s	$\textbf{\color{#d91a1a}-6.93\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.6156ms	1.4308ms	698.8891 Ops/s	845.9581 Ops/s	$\textbf{\color{#d91a1a}-17.38\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	13.7264ms	13.0588ms	76.5769 Ops/s	72.8810 Ops/s	$\textbf{\color{#35bf28}+5.07\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	18.3274ms	16.7766ms	59.6068 Ops/s	58.2880 Ops/s	$\color{#35bf28}+2.26\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	18.1898ms	17.7700ms	56.2747 Ops/s	54.5327 Ops/s	$\color{#35bf28}+3.19\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	18.4674ms	16.9408ms	59.0290 Ops/s	57.6204 Ops/s	$\color{#35bf28}+2.44\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	18.1735ms	17.7412ms	56.3660 Ops/s	55.0828 Ops/s	$\color{#35bf28}+2.33\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	20.3499ms	18.8274ms	53.1140 Ops/s	53.1202 Ops/s	$\color{#d91a1a}-0.01\%$