Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update the Dockerfile with necessary changes from NeMo for H100 self attention accuracy. #550

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Update the Dockerfile with necessary changes from NeMo for H100 self attention accuracy. #550

wants to merge 2 commits into from

Conversation

jstjohn
Copy link
Collaborator

@jstjohn jstjohn commented Dec 19, 2024

  • Noticed poor accuracy on H100 relative to the same reference model involving flash attention.
  • Was able to replicate accuracy on H100 after making these changes to the dockerfile.

@jstjohn
Update the dockerfile with a few necessary changes from NeMo to achie…
4c539f8 
…ve good H100 numerical accuracy in flash attention with TE
@jstjohn
Copy link
Collaborator Author

jstjohn commented Dec 19, 2024

/build-ci

@jstjohn jstjohn requested a review from cspades December 19, 2024 01:19
pstjohn
pstjohn reviewed Dec 19, 2024
View reviewed changes
Dockerfile Outdated Show resolved Hide resolved
@jstjohn
Add UV_NO_CACHE=1 back.
7f39f05 
Signed-off-by: John St. John <[email protected]>
@pstjohn pstjohn mentioned this pull request Dec 19, 2024
Merged
pstjohn added a commit that referenced this pull request Dec 23, 2024
@pstjohn @farhadrgh @jstjohn
Pstjohn/update base image 2410 (#551)
c7281be 
Cherry picks some additional commits into #550 to see if these also fix
CI for 24.10

---------

Signed-off-by: John St. John <[email protected]>
Co-authored-by: Farhad Ramezanghorbani <[email protected]>
Co-authored-by: John St John <[email protected]>
Co-authored-by: John St. John <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pstjohn pstjohn pstjohn left review comments

@dorotat-nv dorotat-nv Awaiting requested review from dorotat-nv dorotat-nv is a code owner

@malcolmgreaves malcolmgreaves Awaiting requested review from malcolmgreaves malcolmgreaves is a code owner

@ohadmo ohadmo Awaiting requested review from ohadmo ohadmo is a code owner

@trvachov trvachov Awaiting requested review from trvachov trvachov is a code owner

@cspades cspades Awaiting requested review from cspades

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jstjohn @pstjohn