Skip to content

Commit

Permalink
update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
mostlygeek committed Dec 9, 2024
1 parent cb978f7 commit 97dae50
Showing 1 changed file with 13 additions and 8 deletions.
21 changes: 13 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,25 @@ Features:

- ✅ Easy to deploy: single binary with no dependencies
- ✅ Single yaml configuration file
-Automatic switching between models
- ✅ Full control over llama.cpp server settings per model
-On-demand model switching
- ✅ Full control over server settings per model
- ✅ OpenAI API support (`v1/completions` and `v1/chat/completions`)
- ✅ Multiple GPU support
- ✅ Run multiple models at once with `profiles`
- ✅ Remote log monitoring at `/log`
- ✅ Automatic unloading of models from GPUs after timeout

## Releases

Builds for Linux and OSX are available on the [Releases](https://github.com/mostlygeek/llama-swap/releases) page.

### Building from source

1. Install golang for your system
1. `git clone [email protected]:mostlygeek/llama-swap.git`
1. `make clean all`
1. Binaries will be in `build/` subdirectory

## config.yaml

llama-swap's configuration is purposefully simple.
Expand Down Expand Up @@ -126,9 +137,3 @@ StartLimitInterval=30
[Install]
WantedBy=multi-user.target
```
## Building from Source
1. Install golang for your system
1. run `make clean all`
1. binaries will be built into `build/` directory

0 comments on commit 97dae50

Please sign in to comment.