An AI API proxy running with Cloudflare worker, supporting multiple AI providers.
- Support APIs provided by OpenAI and other compatible providers
- Fully compliant with the format requirements of OpenAI's API payload and key
- Works with mainstream OpenAI/ChatGPT GUI apps
- Streaming content
- Unique key for users
- Create / delete users
- Reset user's key
- Support for multiple AI providers (e.g., OpenAI, Groq, SambaNova)
- Support for multiple keys for the same provider
- Support for in browser fetch with CORS
- User can reset the key independently
- Time-limited
- Stats for usage
- Clone this repository to your local machine.
- Install npm dependencies.
- Deploy the worker script to Cloudflare. with
npm run deploy
.
This was designed to be used with the AI SDK project.
See AI-Outlook project for an example of how to use it. In particular, see the src/aipane.ts
file.
- Please make sure that the nameservers of your domain is set to the nameservers provided by Cloudflare first. Manual
- Log in to your Cloudflare Dashboard and navigate to the Workers section.
- Click on the "Create a Service" button.
- Input the Service name.
- Keep the "Select a starter" as "Default handler".
- Click on the "Create" button
Now you have the new service created and it shows you the detail of the service.
- Click "Triggers" on the tab bar.
- Click "Add Custom Domain" button.
- Input the domain you want to use, such as
gpt.mydomainname.com
. Don't worry, Cloudflare can automatically configure the proper DNS settings for this. - Click "Add Custom Domain" button to finish the Triggers setting.
Don't leave the detail page and go on.
- Click "Settings" on the tab bar.
- Click "Variables" from the right part.
- In "Environment Variables", Click "Add Variable" button.
- Input two important items. Enable "Encrypt" because they are sensitive.
- Key:
GROQ_API_KEY
, value is your own Groq key. - Key
ACCESS_TOKEN
, value is any random string like a password. Again, both of these pieces of information are very sensitive, so it is strongly recommended to turn on the "Encrypt" option. This way, after you save them, no one will be able to see their values again.
- Key:
- Expand "Workers" in right sidebar.
- Click "KV_AI_PROXY".
- In "Workers KV_AI_PROXY", Click "Create a namespace" button.
- Input new name for the namespace, such as
namespace_gpt
. - Click "Add" button.
- Go back to the detail page of the new created service.
- Go to step 3 of above section, enter "Environment Variables" and scroll down the page.
- In "KV_AI_PROXY Namespace Bindings" section, Click "Add binding" button.
- Input
KV_AI_PROXY
(UPPERCASE) in the left, and choose new KV_AI_PROXY namespace created in step 4. - Click "Save and deploy" button.
- Clone this repository to your local machine.
git clone https://github.com/sctg-development/ai-proxy-cloudflare.git
cd ai-proxy-cloudflare
- Install npm dependencies.
npm install
- Build the worker script.
npm run build
- Open the code of the worker script in
dist/index.js
with a text editor. - Copy all of the code.
- Go back to the detail page of the new created service.
- Click "Quick edit" button at the top right.
- Replace all code with content of pasteboard.
- Click "Save and deploy".
Around one minute later, the new serivce should serve.
Here assume your domain name is ai-proxy.example.com
and the Admin's password (ACCESS_TOKEN
) is Une9f2ijwe
Task | Command |
---|---|
Create new user with name janlay |
curl -H "Authorization: Bearer admin" -X POST https://ai-proxy.example.com/Une9f2ijwe/register/janlay |
Reset user janlay 's key |
curl -H "Authorization: Bearer admin" -X POST https://ai-proxy.example.com/Une9f2ijwe/reset/janlay |
Both of these commands output the user's Key. Please be aware that this key always starts with sk-cfw
and may look like a valid OpenAI Key, but it can only be used for this service.
If you want to delete a user, try this:
curl -H "Authorization: Bearer admin" -X DELETE https://ai-proxy.example.com/Une9f2ijwe/delete/janlay
It's ok if you see "OK".
You can use the proxy for multiple AI providers that use an OpenAI-compatible API format. To add support for a new provider:
curl -H "Authorization: Bearer admin" -X POST https://ai-proxy.example.com/Une9f2ijwe/addkey/host/key
For example, to add support for Groq:
curl -H "Authorization: Bearer admin" -X POST https://ai-proxy.example.com/Une9f2ijwe/addkey/api.groq.com/gsk_your_groq_api_key_here
Here's how to use the service with different providers:
- URL:
https://ai-proxy.example.com/v1/chat/completions
- Headers:
Authorization: Bearer sk-cfw****
(your user key from registration)
- URL:
https://ai-proxy.example.com/openai/v1/chat/completions
- Headers:
Authorization: Bearer sk-cfw****
(your user key from registration)X-Host-Final: api.groq.com
To use with other OpenAI-compatible providers:
- Add the provider's host and API key using the
addkey
command as shown above. - Use the same URL and
Authorization
header as above. - Add the
X-Host-Final
header with the provider's host (e.g.,X-Host-Final: api.someotherprovider.com
).
The proxy will route your request to the specified provider while using your account's API key for that provider.
Here's an example of how to use the proxy with curl for both OpenAI and Groq:
curl https://ai-proxy.example.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk-cfw****" \
-d '{
"model": "gpt-3.5-turbo",
"messages": [{"role": "user", "content": "Say this is a test"}],
"temperature": 0.7
}'
This sample handles the response with brotli
decompression.
curl https://ai-proxy.example.com/openai/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk-cfw****" \
-H "X-Host-Final: api.groq.com" \
-d '{
"model": "llama-3.2-90b-text-preview",
"messages": [{"role": "user", "content": "Say this is a test"}],
"temperature": 0.7
}' --output - | brotli -d
Note: Make sure to replace sk-cfw****
with your actual user key, and use the appropriate model for each provider.
This project uses the MIT license. Please see LICENSE for more information.