IO Bottleneck while loading data

[chnk58hoang]

I'm trying to train DINO ssl with my own dataset (1.2M samples) and now the training process is very very slow although my dataset is stored as shard files. This is my dataset configuration

Is there any idea to speed up my traning process ? I only use 1 gpu nvidia 3090 24gb
Thank you very much !

[JiJiJiang]

Resample to 8kHZ could be cpu-consuming. Try to use more num_workers (e.g. 8 or 16).