Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: update README.md to include set up challenges from virtual env #328

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,14 @@ rm -rf $HOME/.rustup/toolchains/stable-aarch64-apple-darwin
mv $HOME/.rustup/toolchains/1.72.1-aarch64-apple-darwin $HOME/.rustup/toolchains/stable-aarch64-apple-darwin
```

3. If you run into out of memory (OOM) errors, especially while running evaluations that use LLMs as evaluators like toxicity and
3. If you run into the error `ERROR: Cannot install fmeval==0.2.0, fmeval==0.2.1, fmeval==0.3.0, fmeval==0.4.0, fmeval==1.0.0, fmeval==1.0.1, fmeval==1.0.2, fmeval==1.0.3 and fmeval==1.1.0 because these package versions have conflicting dependencies` while installing `fmeval`, please try deactivating and recreating your virtual environment using the steps below. Make sure to replace `<your_virtual_env>` with the name of your actual virtual environment:

```sh
virtualenv --clear <your_virtual_env>
mkvirtualenv <your_virtual_env> -p python3.10
```

4. If you run into out of memory (OOM) errors, especially while running evaluations that use LLMs as evaluators like toxicity and
summarization accuracy, it is likely that your machine does not have enough memory to load the evaluator
models. By default, `fmeval` loads multiple copies of the model into memory to maximize parallelization, where the exact number depends on the number of cores on the machine. To reduce the number of models that get loaded in parallel, you can
set the environment variable `PARALLELIZATION_FACTOR` to a value that suits your machine.
Expand Down
Loading