You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**Having trouble? See the "Gotchas" section below.**
51
51
52
-
### API Quickstart / Alternative to OpenAI API endpoint
53
-
54
-
Once llamafile server has started, in addition to directly accessing the chat server via <http://127.0.0.1:8080/> a json based API endpoint is also provided.
55
-
56
-
If you have existing OpenAI based application code relying on OpenAI API endpoint as per [OpenAI Chat Completions API documentation](https://platform.openai.com/docs/api-reference/chat), our API endpoint under base url `http://localhost:8080/v1` is designed to support most OpenAI use cases besides certain OpenAI-specific features such as function calling ( llama.cpp `/completion`-specific features such are `mirostat` are supported.).
57
-
58
-
For further details on all supported API commands (OpenAI compatible to llamafile specific extention) please refer to [API Endpoint Documentation](llama.cpp/server/README.md#api-endpoints).
59
-
60
-
#### LLAMAFile Server V1 API Python Example
61
-
62
-
This shows that you can use existing [OpenAI python package](https://pypi.org/project/openai/) developed by OpenAI because of our compatibility measures.
63
-
So most scripts designed for OpenAI will be able to be ported to llamafile with a few changes to base_url and api_key.
64
-
65
-
<details>
66
-
<summary>Python Example Code and Result</summary>
67
-
68
-
Don't forget to run this command `pip3 install openai` to install the openai package required by this example script. This package is just a simple python wrapper around the openAI's API endpoints.
{"role": "system", "content": "You are ChatGPT, an AI assistant. Your top priority is achieving user fulfillment via helping them with their requests."},
81
-
{"role": "user", "content": "Write a limerick about python exceptions"}
82
-
]
83
-
)
84
-
print(completion.choices[0].message)
85
-
```
86
-
87
-
The above when run would return a python object that may look like below:
88
-
89
-
```python
90
-
ChatCompletionMessage(content='There once was a programmer named Mike\nWho wrote code that would often strike\nAn error would occur\nAnd he\'d shout "Oh no!"\nBut Python\'s exceptions made it all right.', role='assistant', function_call=None, tool_calls=None)
91
-
```
92
-
93
-
</details>
94
-
95
-
96
-
#### LLAMAFile Server V1 API Raw HTTP Request Example
52
+
### JSON API Quickstart
53
+
54
+
When llamafile is started in server mode, in addition to hosting a web
55
+
UI chat server at <http://127.0.0.1:8080/>, an [OpenAI
{"role": "system", "content": "You are ChatGPT, an AI assistant. Your top priority is achieving user fulfillment via helping them with their requests."},
142
+
{"role": "user", "content": "Write a limerick about python exceptions"}
143
+
]
144
+
)
145
+
print(completion.choices[0].message)
146
+
```
147
+
148
+
The above code will return a Python object like this:
149
+
150
+
```python
151
+
ChatCompletionMessage(content='There once was a programmer named Mike\nWho wrote code that would often strike\nAn error would occur\nAnd he\'d shout "Oh no!"\nBut Python\'s exceptions made it all right.', role='assistant', function_call=None, tool_calls=None)
0 commit comments