lower bounded target q #1330
Open
lower bounded target q #1330
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
le-horizon
commented
May 17, 2022
alf/utils/value_ops.py
Outdated
| @@ -118,6 +118,72 @@ def action_importance_ratio(action_distribution, | |||
| return importance_ratio, importance_ratio_clipped | |||
|
|
|||
|
|
|||
| def generalized_advantage_estimation(rewards, | |||
There was a problem hiding this comment.
I didn't change generalized_advantage_estimation, only moved it closer to action_importance_ratio, git messed up the diff..
Closed
le-horizon
commented
Jun 14, 2022
| improve_w_goal_return: Use return calculated from the distance to hindsight | ||
| goals. Only supports batch_length == 2, one step td. | ||
| improve_w_nstep_bootstrap: Look ahead 2 to n steps, and take the largest | ||
| bootstrapped return to lower bound the value target of the 1st step. |
added 2 commits
June 27, 2022 10:57
…oal distance return, and n-step bootstrapped return)
…ff-policy) algorithms
added 4 commits
June 30, 2022 12:59
le-horizon
commented
Jul 1, 2022
There was a problem hiding this comment.
@emailweixu and @hnyu , I've separated out the HER related logic (the part that move batch_info fields into alg_info and loss calculation) into a new file her_algorithms.
Let me know how you like this version.
Thanks,
Le
| improve_w_goal_return: Use return calculated from the distance to hindsight | ||
| goals. Only supports batch_length == 2, one step td. | ||
| improve_w_nstep_bootstrap: Look ahead 2 to n steps, and take the largest | ||
| bootstrapped return to lower bound the value target of the 1st step. |
added 2 commits
July 1, 2022 09:37
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
minimum change for lower bounded value target (for episodic return, goal distance return, and n-step bootstrapped return)