Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【TST在windows10上运行错误】RuntimeError: CUDA out of memory. #1

Open
bettermorn opened this issue Jan 19, 2022 · 0 comments
Open
Assignees
Labels
question Further information is requested

Comments

@bettermorn
Copy link
Owner

系统环境及依赖包

os : Windows-10-10.0.19042-SP0
python : 3.8.12
tsai : 0.2.24
fastai : 2.5.3
fastcore : 1.3.27
torch : 1.10.1
n_cpus : 8
device : cuda (NVIDIA GeForce MX150)

使用TST模型训练时会出现以下错误

RuntimeError: CUDA out of memory. Tried to allocate 764.00 MiB (GPU 0; 2.00 GiB total capacity; 841.93 MiB already allocated; 277.44 MiB free; 854.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

原因及解决方法如下

  1. 减小模型训练的batch_size
  2. kill 占用GPU内存的进程
  3. 增加语句,清除缓存
  • 设置 max_split_size_mb
    设置环境变量PYTORCH_CUDA_ALLOC_CONF为max_split_size_mb:128
@bettermorn bettermorn added the question Further information is requested label Jan 19, 2022
@bettermorn bettermorn self-assigned this Jan 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant