Skip to content

Commit

Permalink
Merge branch 'main' into development
Browse files Browse the repository at this point in the history
  • Loading branch information
@fractalego
fractalego authored Dec 23, 2023
2 parents a040597 + 92f30c4 commit d1d6f48
Showing 1 changed file with 3 additions and 7 deletions.
10 changes: 3 additions & 7 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,12 +46,8 @@ Please see the examples in the following chapters.


## LLM side (needs a GPU)
The second part is a machine that runs on a machine accessible from the interface side.
The initial configuration is for a local deployment of language models.
No action is needed to run WAFL if you want to run it as a local instance.

However, a multi-user setup will benefit for a dedicated server.
In this case, a docker image can be used
The second part (LLM side) is a model server for the speech-to-text model, the LLM, the embedding system, and the text-to-speech model.
A docker image can be used to run it as in the following:

```bash
$ docker run -p8080:8080 --env NVIDIA_DISABLE_REQUIRE=1 --gpus all fractalego/wafl-llm:0.80
Expand Down Expand Up @@ -97,4 +93,4 @@ Documentation
The documentation can be found at [wafl.readthedocs.io](https://wafl.readthedocs.io).


<a rel="me" href="https://fractalego.social/@wafl">Mastodon</a>
<a rel="me" href="https://fractalego.social/@wafl">Mastodon</a>

0 comments on commit d1d6f48

Please sign in to comment.