Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update README and model.py for Nvidia GPU support #530

Merged
merged 1 commit into from
Jan 3, 2025

Conversation

ericcurtin
Copy link
Collaborator

@ericcurtin ericcurtin commented Jan 3, 2025

README.md: Updated the Nvidia GPU (cuda) status to ✅ indicating support. model.py: Added logic to handle CUDA_VISIBLE_DEVICES, returning the corresponding quay.io/ramalama/cuda:latest image.

Summary by Sourcery

New Features:

  • Nvidia GPUs are now supported.

Copy link
Contributor

sourcery-ai bot commented Jan 3, 2025

Reviewer's Guide by Sourcery

This pull request adds support for Nvidia GPUs by updating the model.py file to handle CUDA_VISIBLE_DEVICES and updating the README.md file to reflect the new support.

Sequence diagram for GPU type detection and image selection

sequenceDiagram
    participant Client
    participant Model
    participant GPUDetector

Client->>Model: _image(args)
Model->>GPUDetector: get_gpu()
GPUDetector-->>Model: Return gpu_type
Note over Model: Check GPU type
alt gpu_type is HIP_VISIBLE_DEVICES
    Model-->>Client: Return rocm image
else gpu_type is CUDA_VISIBLE_DEVICES
    Model-->>Client: Return cuda image
else gpu_type is ASAHI_VISIBLE_DEVICES
    Model-->>Client: Return asahi image
else no specific GPU type
    Model-->>Client: Return default image
end
Loading

File-Level Changes

Change Details Files
Added support for Nvidia GPUs.
  • Added logic to handle CUDA_VISIBLE_DEVICES and return quay.io/ramalama/cuda:latest image.
ramalama/model.py
Updated documentation to reflect Nvidia GPU support.
  • Updated Nvidia GPU status to ✅ indicating support.
README.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time. You can also use
    this command to specify where the summary should be inserted.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@ericcurtin
Copy link
Collaborator Author

Once we merge:

#452

and we rebuild all the containers, this can go in.

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @ericcurtin - I've reviewed your changes - here's some feedback:

Overall Comments:

  • Please use elif instead of else if in model.py - this is the correct Python syntax
Here's what I looked at during the review
  • 🟢 General issues: all looks good
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟢 Complexity: all looks good
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

README.md: Updated the Nvidia GPU (cuda) status to
:white_check_mark: indicating support. model.py: Added logic to
handle CUDA_VISIBLE_DEVICES, returning the corresponding
quay.io/ramalama/cuda:latest image.

Signed-off-by: Eric Curtin <[email protected]>
@ericcurtin ericcurtin force-pushed the we-publish-cuda-now branch from 3dcd1d5 to 337942c Compare January 3, 2025 13:43
@rhatdan
Copy link
Member

rhatdan commented Jan 3, 2025

LGTM

@rhatdan rhatdan merged commit 0a728ed into main Jan 3, 2025
11 checks passed
@ericcurtin ericcurtin deleted the we-publish-cuda-now branch January 3, 2025 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants