Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore unused parameters when using DDP + PPG #1117

Merged
merged 1 commit into from
Dec 8, 2021

Conversation

breakds
Copy link
Contributor

@breakds breakds commented Dec 5, 2021

Motivation

To make DDP + PPG work, even with #1116, DDP will still panic about unused parameters. Note that there are 2 isolated problems:

  1. Parameters contributes to output, but the output is not used in computing the loss
  2. Parameters that does not even contribute to the output

#1116 aims to fix No.1, and we need to instruct DDP to proactively prune the unused parameters. Though this will adds a little overhead but it is necessary for PPG.

Solution

Add a globally configurable parameters to find and ignore unused parameters for DDP. By default it is disabled but for PPG it is explicitly enabled.

Testing

Tested together with #1116 on ac/ppo/ppg cart pole to make sure all of them trains correct. PPG procgen is trained and reported in #1099.

@breakds breakds added the enhancement New feature or request label Dec 5, 2021
@breakds breakds force-pushed the PR/breakds/ppg_find_unused_parameters branch from ab330be to 7a4e8da Compare December 6, 2021 18:31
@breakds breakds mentioned this pull request Dec 6, 2021
15 tasks
emailweixu
emailweixu previously approved these changes Dec 7, 2021
Base automatically changed from PR/breakds/condition_data_distributed to pytorch December 7, 2021 20:39
@breakds
Copy link
Contributor Author

breakds commented Dec 7, 2021

Rebased pytorch.

@emailweixu
Copy link
Contributor

Rebased pytorch.

Please avoid rebase in the future. As it's unclear what is changed. Use pull without --rebase instead.

@breakds breakds merged commit 0b4ec5b into pytorch Dec 8, 2021
@breakds breakds deleted the PR/breakds/ppg_find_unused_parameters branch December 8, 2021 04:24
pd-perry pushed a commit to pd-perry/alf that referenced this pull request Dec 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants