gprc api
#2101
Replies: 1 comment
-
Llama.cpp has a simple HTTP API server There are a few other Bindings mentioned on the README.md homepage of this repo. I use the Python API by Abetlen et all abetlen/llama-cpp-python.. which very conveniently mimics the OpenAI API... so you can use any of the OpenAI ChatGPT clients >> llama-cpp-python >> llama.cpp. Sorry, I don't of a GRPC API for llama.cpp |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
There is any repo like "llama-server" or similar? so we can have a llama.cpp instance running a model and a GRPC api for sending prompts and receiving completions?
Beta Was this translation helpful? Give feedback.
All reactions