Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Training time for 400k iterations #17

Closed
caide199212 opened this issue Feb 16, 2023 · 2 comments
Closed

Training time for 400k iterations #17

caide199212 opened this issue Feb 16, 2023 · 2 comments

Comments

@caide199212
Copy link

Nice work! And I'am trying to reproduce the training with Reds dataset on 4 GPU(each with 16G memory). The batchsize is set 1 as suggested in the config file. The total training time looks to be ~31 days. I'm wondering if this is case for your training or I'm wrong somewhere.

Thanks a lot.

@ericzw
Copy link
Member

ericzw commented Feb 17, 2023

We use 8 V100 GPUs with 32GB.
The performance is depended on the sequence length you used for training (1-100).
with 20imgs/sequence, it cost about 7 days for training.

@caide199212
Copy link
Author

Got it. Thank you very much for your prompt reply.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants