-
Notifications
You must be signed in to change notification settings - Fork 706
Add support for Intel Gaudi/HPU backend #1662
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for Intel Gaudi/HPU backend #1662
Conversation
@matthewdouglas @jiqing-feng Please help review this PR. We are working on documentation and unit-tests and plan to add those in separate follow-up PRs. Thanks. |
@@ -442,7 +442,7 @@ def __init__( | |||
) | |||
# self.persistent_buffers = [] # TODO consider as way to save quant state | |||
self.compute_dtype = compute_dtype | |||
self.compute_type_is_set = False | |||
self.compute_type_is_set = False if compute_dtype is None else True |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change seems not related to hpu, did you met any block issues?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jiqing-feng Please check description in this PR #1623 . Issue which gets fixed with this change is described there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. Thanks!
9fb71c1
to
068f0e1
Compare
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, just a couple small nits! Thanks!
updates the assertion message Co-authored-by: Matthew Douglas <[email protected]>
Co-authored-by: Matthew Douglas <[email protected]>
Fix lint issue
This PR enables the support of bitsandbytes for HPU (Intel Gaudi) devices.
These changes add support for single and double NF4 quantization/dequantization using Intel Gaudi hardware.