Skip to content

Commit 3ec9069

Browse files
authored
Add step to fix NCCL error with default environment
1 parent 8d7c593 commit 3ec9069

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
This repository provides an easy-to-use solution to run inference servers on [Slurm](https://slurm.schedmd.com/overview.html)-managed computing clusters using [vLLM](https://docs.vllm.ai/en/latest/). All scripts in this repository runs natively on the Vector Institute cluster environment, and can be easily adapted to other environments.
33

44
## Installation
5-
If you are using the Vector cluster environment, and you don't need any customization to the inference server environment, you can skip this step and go to the next section. Otherwise, you might need up to 10GB of storage to setup your own virtual environment. The following steps needs to be run only once for each user.
5+
If you are using the Vector cluster environment, and you don't need any customization to the inference server environment, all you need to do is run `pip install vllm-nccl-cu12` and go to the next section. Otherwise, you might need up to 10GB of storage to setup your own virtual environment. The following steps needs to be run only once for each user.
66

77
1. Setup the virtual environment for running inference servers, run
88
```bash

0 commit comments

Comments
 (0)