Ignore unused parameters when using DDP + PPG #1117
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
breakds
force-pushed
the
PR/breakds/ppg_find_unused_parameters
branch
from
ab330be
to
7a4e8da
Compare
emailweixu
previously approved these changes
Dec 7, 2021
Base automatically changed from
PR/breakds/condition_data_distributed
to
pytorch
December 7, 2021 20:39
breakds
force-pushed
the
PR/breakds/ppg_find_unused_parameters
branch
from
7a4e8da
to
f822776
Compare
|
Rebased |
Please avoid rebase in the future. As it's unclear what is changed. Use pull without --rebase instead. |
emailweixu
approved these changes
Dec 8, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
To make DDP + PPG work, even with #1116, DDP will still panic about unused parameters. Note that there are 2 isolated problems:
#1116 aims to fix No.1, and we need to instruct DDP to proactively prune the unused parameters. Though this will adds a little overhead but it is necessary for PPG.
Solution
Add a globally configurable parameters to find and ignore unused parameters for DDP. By default it is disabled but for PPG it is explicitly enabled.
Testing
Tested together with #1116 on ac/ppo/ppg cart pole to make sure all of them trains correct. PPG procgen is trained and reported in #1099.