Skip to content

Add support for Intel Gaudi/HPU backend #1662

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 5, 2025

Conversation

rsshaik1
Copy link
Contributor

This PR enables the support of bitsandbytes for HPU (Intel Gaudi) devices.

  • Adds HPU as the supported device.
  • Creates a backend for HPU devices (bitsandbytes/backends/hpu).

These changes add support for single and double NF4 quantization/dequantization using Intel Gaudi hardware.

@rsshaik1 rsshaik1 changed the title supports hpu backend in main branch Add support for Intel Gaudi/HPU backend May 29, 2025
@vivekgoe
Copy link

@matthewdouglas @jiqing-feng Please help review this PR. We are working on documentation and unit-tests and plan to add those in separate follow-up PRs. Thanks.

@@ -442,7 +442,7 @@ def __init__(
)
# self.persistent_buffers = [] # TODO consider as way to save quant state
self.compute_dtype = compute_dtype
self.compute_type_is_set = False
self.compute_type_is_set = False if compute_dtype is None else True
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change seems not related to hpu, did you met any block issues?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jiqing-feng Please check description in this PR #1623 . Issue which gets fixed with this change is described there.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Thanks!

@matthewdouglas matthewdouglas added this to the v0.47.0 milestone Jun 2, 2025
@matthewdouglas matthewdouglas self-requested a review June 2, 2025 16:23
@rsshaik1 rsshaik1 force-pushed the Gaudi_support branch 2 times, most recently from 9fb71c1 to 068f0e1 Compare June 3, 2025 09:11
Copy link

github-actions bot commented Jun 4, 2025

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

matthewdouglas
matthewdouglas previously approved these changes Jun 4, 2025
Copy link
Member

@matthewdouglas matthewdouglas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a couple small nits! Thanks!

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
updates the assertion message

Co-authored-by: Matthew Douglas <[email protected]>
rsshaik1 and others added 3 commits June 5, 2025 10:27

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Co-authored-by: Matthew Douglas <[email protected]>

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fix lint issue

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@matthewdouglas matthewdouglas merged commit 812ef06 into bitsandbytes-foundation:main Jun 5, 2025
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants