Go LLM Proxy

A secure proxy server to call local LLMs running in Ollama, LMStudio, Llamacpp, etc. using a single endpoint. This proxy supports key management, request logging, and monitoring.

FOR DEVELOPMENT ONLY. DO NOT USE FOR PRODUCTION (NOT YET READY)

Features

• API Key Management: Create, update, delete, and regenerate API keys
• Admin API Security: Protect management APIs with environment-configured API key
• Route Configuration: Configure multiple target URLs with path prefixes
• Secure Proxy: Forward requests to target LLM APIs with authentication
• HTTPS Support: Built-in TLS with Let's Encrypt or self-signed certificates
• Request Logging: Log all requests and responses to SQLite database
• Web UI: Beautiful admin dashboard for monitoring and management
• Swagger Documentation: Auto-generated API documentation with authentication
• Statistics: Usage statistics and analytics
• Rate Limiting: Built-in rate limiting per API key
• CORS Support: Configurable CORS policies

Quick Start

Prerequisites

• Go 1.21 or higher
• SQLite 3

Installation

1. Clone the repository:

git clone <repository-url>
cd go-llm-proxy

2. Install dependencies:

make setup

3. Configure security (recommended):

# Copy the example environment file
cp .env.example .env

# Generate a secure admin API key
openssl rand -base64 32

# Edit .env and set ADMIN_API_KEY with the generated key

4. Run the server:

make run

The proxy will start on http://localhost:8080

Using the Proxy

1. Web UI: Open http://localhost:8080 in your browser
2. API Documentation: Visit http://localhost:8080/swagger/index.html
   • Click the "Authorize" button and enter your ADMIN_API_KEY to test endpoints
3. Health Check: GET http://localhost:8080/health

Note: If you set ADMIN_API_KEY, all API endpoints require authentication.

🔒 Run with Local HTTPS (Optional)

For local development with HTTPS:

# Use the helper script
./scripts/run-https-local.sh

# Or manually
export TLS_ENABLED=true
export TLS_HTTPS_PORT=8443
make dev

Then open https://localhost:8443 (accept the security warning for self-signed certificate)

Security

Admin API Key Authentication

All management API endpoints (/api/*) can be protected with an admin API key:

# Generate a secure key
openssl rand -base64 32

# Set it as an environment variable
export ADMIN_API_KEY="your-generated-key"

When ADMIN_API_KEY is set, all API requests must include the key:

# Using X-API-Key header
curl -H "X-API-Key: your-key" http://localhost:8080/api/keys

# Using Authorization header
curl -H "Authorization: Bearer your-key" http://localhost:8080/api/keys

Swagger UI Authentication:

1. Open http://localhost:8080/swagger/index.html
2. Click the Authorize button (🔒)
3. Enter your admin API key
4. Test endpoints

Protected Endpoints:

• All /api/keys/* - API key management
• All /api/logs/* - Request logs
• All /api/stats - Statistics
• All /api/route-configs/* - Route configuration
• All /api/headers/* - Header management

Public Endpoints:

• /health - Health check
• /swagger/* - API documentation
• / and /ui - Web interface

⚠️ Important: For production, always set a strong ADMIN_API_KEY. If not set, APIs are accessible without authentication (development mode only).

Configuration

The proxy can be configured using environment variables:

Server Configuration

• SERVER_HOST: Server host (default: "0.0.0.0")
• SERVER_PORT: Server port (default: 8080)

Database Configuration

• DB_PATH: Path to SQLite database file (default: "./proxy.db")

Logging Configuration

• LOG_LEVEL: Log level (default: "info")
• LOG_ENABLE_REQUEST_LOG: Enable request logging (default: true)
• LOG_RETENTION_DAYS: Log retention period in days (default: 30)

Security Configuration

• ADMIN_API_KEY: Admin API key for management endpoints (no default, recommended for production)
• CORS_ENABLED: Enable CORS (default: true)
• CORS_ALLOWED_ORIGINS: Allowed CORS origins (default: "*")

TLS/HTTPS Configuration

• TLS_ENABLED: Enable HTTPS (default: false)
• TLS_DOMAIN: Domain for Let's Encrypt (empty = use self-signed certs)
• TLS_CERT_FILE: Path to certificate file for self-signed (default: "./cert.pem")
• TLS_KEY_FILE: Path to key file for self-signed (default: "./key.pem")
• TLS_CACHE_DIR: Certificate cache directory (default: "./cert-cache")
• TLS_HTTP_PORT: HTTP port for ACME challenge (default: 80)
• TLS_HTTPS_PORT: HTTPS port (default: 443)
• RATE_LIMITING_ENABLED: Enable rate limiting (default: true)

API Usage

Authentication

Use either of these methods to authenticate requests:

1. Authorization Header:

   Authorization: Bearer your-api-key
   

2. X-API-Key Header:

   X-API-Key: your-api-key
   

Example Request

After configuring routes in the Web UI, use paths like:

curl -X POST http://localhost:8080/ollama/api/generate \
  -H "Authorization: Bearer your-api-key" \
  -H "Content-Type: application/json" \
  -d '{"model": "llama2", "prompt": "Hello!"}'

curl -X POST http://localhost:8080/openai/v1/chat/completions \
  -H "Authorization: Bearer your-api-key" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4", "messages": [{"role": "user", "content": "Hello!"}]}'

API Endpoints

API Key Management

• POST /api/keys - Create new API key
• GET /api/keys - List all API keys
• GET /api/keys/{id} - Get specific API key
• PUT /api/keys/{id} - Update API key
• DELETE /api/keys/{id} - Delete API key
• POST /api/keys/{id}/regenerate - Regenerate API key

Request Logs

• GET /api/logs - Get request logs with filtering
• GET /api/logs/{id} - Get specific request log
• DELETE /api/logs/cleanup - Clean old logs

Statistics

• GET /api/stats - Get usage statistics

Route Configuration

• GET /api/route-configs - List all route configurations
• POST /api/route-configs - Create new route configuration
• GET /api/route-configs/{id} - Get specific route configuration
• PUT /api/route-configs/{id} - Update route configuration
• DELETE /api/route-configs/{id} - Delete route configuration

Development

Project Structure

go-llm-proxy/
├── cmd/
│   └── server/          # Main application entry point
├── internal/
│   ├── config/          # Configuration management
│   ├── database/        # Database setup and connection
│   ├── handlers/        # HTTP handlers
│   ├── middleware/      # HTTP middleware
│   ├── models/          # Data models
│   └── services/        # Business logic services
├── web/
│   ├── static/          # Static files (CSS, JS)
│   └── templates/       # HTML templates
├── docs/                # Swagger documentation
├── go.mod
├── go.sum
├── Makefile
└── README.md

Development Commands

# Run in development mode
make dev

# Run tests
make test

# Generate Swagger documentation
make generate-swagger

# Build for production
make build-prod

# Clean build artifacts
make clean

Generating Swagger Documentation

1. Install swag:

go install github.com/swaggo/swag/cmd/swag@latest

2. Generate documentation:

make generate-swagger

Database Schema

The application uses SQLite with the following main tables:

• api_keys: Stores API key information
• request_logs: Stores all HTTP request/response logs
• route_configs: Stores route configuration for path-based routing

Deployment

Docker Deployment

1. Build the Docker image:

docker build -t go-llm-proxy .

2. Run the container:

docker run -p 8080:8080 -v $(pwd)/data:/app/data go-llm-proxy

Production Deployment

1. Build the production binary:

make build-prod

2. Set environment variables:

export SERVER_HOST=0.0.0.0
export SERVER_PORT=8080
export DB_PATH=/data/proxy.db

3. Configure routes through the Web UI at http://localhost:8080

4. Run the binary:

./bin/proxy

Security Considerations

• API keys are generated using cryptographically secure random bytes
• All requests are logged for audit purposes
• CORS is configurable
• Rate limiting can be enabled per API key
• Database should be properly secured in production

Monitoring

The proxy provides comprehensive monitoring through:

• Web Dashboard: Real-time statistics and logs
• API Endpoints: Programmatically access statistics
• Request Logging: Detailed request/response logs
• Health Checks: Service health monitoring

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Submit a pull request

License

This project is licensed under the Apache License 2.0.