GPU memory requirements for training

[Sunappnio]

This model is huge and takes a lot of gpu memory, so batch size can only be set a small number. May I ask which gpu do you use?

[Cryolite]

I use RTX-3090, so the GPU memory is 24GB.

The speed of learning on GPUs is achieved through parallelism with a certain batch size, so learning with small batch sizes is likely to be extremely slow.

If you are training on a low-memory GPU, try the --checkpointing option. This option slows down the training speed by about 30% to 40%, but it dramatically reduces the GPU memory requirement and allows training with large batch sizes, which may result in faster training.

For more information on what checkpointing (do not confuse this with snapshots, which store progresses of learning) does, see for example https://pytorch.org/docs/stable/checkpoint.html.

[Sunappnio]

Thanks! Finally I rent some V100s, and play around supervised learning, your model works like a charm, better than cnn