Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update CUDA install instructions for generate() API #23155

Open
wants to merge 1 commit into
base: gh-pages
Choose a base branch
from
Open

Update CUDA install instructions for generate() API #23155

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 4 additions & 6 deletions docs/genai/howto/install.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,18 +38,16 @@ The CUDA toolkit can be downloaded from the [CUDA Toolkit Archive](https://devel

Ensure that the `CUDA_PATH` environment variable is set to the location of your CUDA installation.

#### CUDA 11

```bash
pip install onnxruntime-genai-cuda --index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-11/pypi/simple/
```

#### CUDA 12

```bash
pip install onnxruntime-genai-cuda
```

#### CUDA 11

To use this API with CUDA 11, you need to [build and install from source](build-from-source.md).


## Nuget package installation

Expand Down
5 changes: 1 addition & 4 deletions docs/genai/tutorials/phi3-python.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,6 @@ Are you on a Windows machine with GPU?
2. Install the generate() API

```
pip install numpy
pip install --pre onnxruntime-genai-directml
```

Expand Down Expand Up @@ -125,8 +124,7 @@ Are you on a Windows machine with GPU?
2. Install the generate() API

```
pip install numpy
pip install --pre onnxruntime-genai-cuda --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-genai/pypi/simple/
pip install --pre onnxruntime-genai-cuda
```

3. Run the model
Expand Down Expand Up @@ -159,7 +157,6 @@ Are you on a Windows machine with GPU?
2. Install the generate() API for CPU

```
pip install numpy
pip install --pre onnxruntime-genai
```

Expand Down
8 changes: 1 addition & 7 deletions docs/genai/tutorials/phi3-v.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,13 +102,7 @@ You can download the models here:

3. Install the generate() API

* CUDA 11

```bash
pip install onnxruntime-genai-cuda --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-11/pypi/simple/
```

* CUDA 12
Note: this package uses CUDA 12. To use CUDA 11, you need to [build and install from source](../howto/build-from-source.md).

```bash
pip install onnxruntime-genai-cuda
Expand Down
Loading