Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix init_value and end_value in cosine decay #884

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fix init_value and end_value in cosine decay #884

wants to merge 1 commit into from

Commits on Mar 26, 2024

  1. Fix init_value and end_value in cosine decay 

    Problem:
* the starting value was not always init_value
* the last value was not always end_value

Solution:
* changed the formulas (they are now compatible with the pytorch implementation https://pytorch.org/docs/stable/generated/torch.optim.lr_scheduler.CosineAnnealingLR.html)
* added test to check that starting and end value coincide with `init_value` and `end_value` respectively

Misc:
   renamed alpha -> end_value in warmup_cosine_decay_schedule
PiperOrigin-RevId: 619109148
    @fabianp
    fabianp authored and OptaxDev committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    cb6cef2 View commit details
    Browse the repository at this point in the history