Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Transformer Engine's version of RMSNorm and LayerNorm #1235

Closed
wants to merge 7 commits into from
Closed

Add Transformer Engine's version of RMSNorm and LayerNorm #1235

wants to merge 7 commits into from

Commits on Jun 10, 2024

  1. added option for te version of norms

    lintangsutawika committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    4705db4 View commit details
    Browse the repository at this point in the history

  2. import TERMSNorm

    @lintangsutawika
    lintangsutawika committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    d982b29 View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. add te norm options to norm arg

    @lintangsutawika
    lintangsutawika committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    cf40c0d View commit details
    Browse the repository at this point in the history

  2. add TE objects in weight decay function

    @lintangsutawika
    lintangsutawika committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    e45f16f View commit details
    Browse the repository at this point in the history

  3. reformat

    @lintangsutawika
    lintangsutawika committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    8cfb078 View commit details
    Browse the repository at this point in the history

  4. add TERMSNorm and TELayerNorm

    @lintangsutawika
    lintangsutawika committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    217b4c5 View commit details
    Browse the repository at this point in the history

  5. Update NeoXArgs docs automatically

    github-actions committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    2a290ca View commit details
    Browse the repository at this point in the history