Pretrained model performance is low.

[minju-Kang]

Hi, I run pretraining code with split_0.01_2 (same with 3DIoUMatch), but I think the performance is much lower than the numbers mentioned on your paper.
I used pv_rcnn.yaml for configuration file, and trained 80 epochs on one GPU.

Car [email protected], 0.70, 0.70:
bbox AP:90.0876, 80.0455, 78.3626
bev AP:88.7344, 78.5149, 70.2348
3d AP:83.6766, 67.3691, 59.6103
aos AP:89.15, 78.45, 76.29
Car [email protected], 0.70, 0.70:
bbox AP:95.5903, 84.4095, 79.1778
bev AP:91.8627, 80.6292, 73.5810
3d AP:83.8539, 68.1927, 61.4822
aos AP:94.45, 82.45, 76.96
Car [email protected], 0.50, 0.50:
bbox AP:90.0876, 80.0455, 78.3626
bev AP:90.2690, 88.1668, 80.0592
3d AP:90.2208, 87.1712, 79.6355
aos AP:89.15, 78.45, 76.29
Car [email protected], 0.50, 0.50:
bbox AP:95.5903, 84.4095, 79.1778
bev AP:95.8997, 89.8074, 82.6548
3d AP:95.8319, 87.3881, 82.1202
aos AP:94.45, 82.45, 76.96
Pedestrian [email protected], 0.50, 0.50:
bbox AP:44.3563, 37.3375, 33.0982
bev AP:43.0798, 35.9866, 31.8159
3d AP:35.3727, 28.8929, 27.9237
aos AP:29.54, 25.10, 22.97
Pedestrian [email protected], 0.50, 0.50:
bbox AP:40.5246, 34.1630, 31.5183
bev AP:39.4852, 31.5727, 28.9043
3d AP:33.9902, 27.9883, 24.5065
aos AP:24.06, 19.80, 18.50
Pedestrian [email protected], 0.25, 0.25:
bbox AP:44.3563, 37.3375, 33.0982
bev AP:47.1612, 39.6489, 38.8192
3d AP:47.1057, 39.5440, 38.6939
aos AP:29.54, 25.10, 22.97
Pedestrian [email protected], 0.25, 0.25:
bbox AP:40.5246, 34.1630, 31.5183
bev AP:45.9999, 38.4472, 34.6189
3d AP:45.9230, 37.2270, 34.5071
aos AP:24.06, 19.80, 18.50
Cyclist [email protected], 0.50, 0.50:
bbox AP:21.4094, 15.6852, 16.0712
bev AP:19.8382, 11.0331, 10.9091
3d AP:12.6055, 9.2240, 8.9105
aos AP:14.14, 10.18, 10.66
Cyclist [email protected], 0.50, 0.50:
bbox AP:18.6879, 11.4710, 11.5689
bev AP:16.2348, 9.5348, 9.2544
3d AP:11.2959, 6.8327, 6.7031
aos AP:12.39, 7.48, 7.74
Cyclist [email protected], 0.25, 0.25:
bbox AP:21.4094, 15.6852, 16.0712
bev AP:21.3333, 15.5184, 15.6056
3d AP:21.3333, 15.4820, 15.5694
aos AP:14.14, 10.18, 10.66
Cyclist [email protected], 0.25, 0.25:
bbox AP:18.6879, 11.4710, 11.5689
bev AP:18.7750, 11.5075, 11.2206
3d AP:18.7750, 11.4757, 11.1858
aos AP:12.39, 7.48, 7.74

Could you share your performance on different split sets?
Also, there are many pv_rcnn configuration file on your code. Which one did you used?

[Whale-ice]

Hi,thanks for your interest in my work, I uploaded the new code, the previous code had some problems (I mentioned in the readme), you can try in the new code.
I provided pv_rcnn_ssl_db.yaml , log.txt for 5% ,10% and the split.txt. you can find them in logfile. You can take it as a reference, I didn't set any special parameters.
I will release my log.txt for 1% and 2% as soon as possible (about three weeks as I am currently out of school and cannot connect to the server)

[minju-Kang]

Thanks! It will be helpful.
Another question, did you include planes data for training?