Skip to content

Commit

Permalink
added fastapi-server instrucs
Browse files Browse the repository at this point in the history
  • Loading branch information
pchalasani committed Feb 29, 2024
1 parent 24a1f1c commit ffdcdfd
Show file tree
Hide file tree
Showing 5 changed files with 91 additions and 22 deletions.
1 change: 1 addition & 0 deletions fastapi-server/.dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@
.venv
poetry.lock
pyproject.toml
README.md

2 changes: 1 addition & 1 deletion fastapi-server/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM --platform=linux/amd64 python:3.11
FROM python:3.11


# Set the working directory in the container
Expand Down
15 changes: 10 additions & 5 deletions fastapi-server/Makefile
Original file line number Diff line number Diff line change
@@ -1,14 +1,19 @@
LOCAL_ARCH=$(shell uname -m)
GCLOUD_ARCH=amd64
IMAGE_NAME=langroid-server
TAG=latest

server:
docker build -t server .
docker build --platform linux/$(LOCAL_ARCH) -t $(IMAGE_NAME):$(TAG) .

run:
docker run -d -p 80:80 server
docker run --env-file .env -d -p 80:80 $(IMAGE_NAME):$(TAG)

stop:
docker stop $(shell docker ps -q)

gbuild:
docker build -t gcr.io/langroid/langroid-server:v1 .
gserver:
docker build --platform=linux/$(GCLOUD_ARCH) -t gcr.io/langroid/$(IMAGE_NAME):$(TAG) .

gpush:
docker push gcr.io/langroid/langroid-server:v1
docker push gcr.io/langroid/$(IMAGE_NAME):$(TAG)
60 changes: 47 additions & 13 deletions fastapi-server/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,16 @@
# Setup a REST API server on GCP for a Langroid script

Ensure that your env secrets are in this folder as a `.env` file.
We will of course *not* include it in the docker image
(which is why it's in the `.dockerignore` file) but we will use it
for local testing by passing this file in as an env file to the uvicorn server.
(See the defn of `make run` in the Makefile.)

## local testing

Build server using `make server`, run it with `make run`
Build server using `make server`, run it with `make run`.
See the definitions of these in the Makefile. Notice that
we are passing in the env variables

## Curl examples to test the server locally

Expand All @@ -25,6 +33,13 @@ curl -X POST \
```
(If you omit the `-o out.txt` part, the response will be printed to the terminal.)

`process_file_and_data` endpoint:

```bash
curl -X POST "http://localhost:8000/process_file_and_data/" \
-F "file=@/tmp/query.txt" \
-F 'complex_data={"id": 1, "item": {"name": "Item Name", "description": "A description", "quantity": 10, "tags": ["tag1", "tag2"]}, "related_items": [{"name": "Related Item 1", "description": "Description 1", "quantity": 5, "tags": ["tag3", "tag4"]}, {"name": "Related Item 2", "description": "Description 2", "quantity": 3, "tags": ["tag5", "tag6"]}]}'
```

## Deploy to google cloud run

Expand All @@ -36,20 +51,12 @@ See other details here:
https://chat.openai.com/share/c34583c8-b88e-4a70-bf24-83229700c020


Run these from within the dir where the Dockerfile is located:

```bash
gcloud auth configure-docker
docker build -t gcr.io/langroid/langroid-server:v1 .
docker push gcr.io/langroid/langroid-server:v1
```

The `build` and `push` cmds are also available via the Makefile
as `make gbuild` and `make gpush` respectively.
Run `make gserver`, `make gpush` from within the dir where the Dockerfile is
located.

Go to Google Cloud Run Service and create a new service,
selecting the latest version of the pushed docker image above:
`gcr.io/langroid/langroid-server:v1`
selecting the latest version of the pushed docker image above, e.g.:
`gcr.io/langroid/langroid-server:latest`

When setting up the service:
- ensure you select the same port number as in the Dockerfile, e.g. 80.
Expand All @@ -61,6 +68,33 @@ If the service fails to start due to an error like `uvicorn: exec format error`,
then you may be able to fix it by explicitly choosing an architecture
in the Dockerfile, e.g. `linux/amd64` (which we chose in the Dockerfile).

### Creating secrets in google cloud
Some commands for quick reference:

```bash
gcloud secrets create openai-api-key --replication-policy="automatic"
echo -n "your-openai-api-key" | gcloud secrets versions add openai-api-key --data-file=-
```

After creating your secret and adding its value, you may need to set appropriate
permissions for the secret. Use gcloud secrets add-iam-policy-binding to grant access
to the secret:

```bash
gcloud secrets add-iam-policy-binding openai-api-key \
--member="serviceAccount:[email protected]" \
--role="roles/secretmanager.secretAccessor"
```

To expose one of these as environment var named `OPENAI_API_KEY` in the cloud run
service:

```bash
gcloud run services update langroid-server \
--update-secrets OPENAI_API_KEY=openai-api-key:latest \
--region=us-east4
```

## Test GCP endpoints

Same curl cmds as above, but use the endpoint url from the GCP Cloud Run service,
Expand Down
35 changes: 32 additions & 3 deletions fastapi-server/app.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from typing import Any
from fastapi import FastAPI, File, UploadFile
from typing import Any, Dict, List
from fastapi import FastAPI, File, UploadFile, Form
from fastapi.responses import FileResponse
from pydantic import BaseModel
from pydantic import BaseModel, Json
import uvicorn
import os
import langroid as lr
Expand All @@ -10,6 +10,19 @@
class TextInput(BaseModel):
text: str

# Define your Pydantic models for the complex payload
class Item(BaseModel):
name: str
description: str
quantity: int
tags: List[str]

class ComplexItem(BaseModel):
id: int
item: Item
related_items: List[Item]


class Server:
def __init__(self):
self.setup()
Expand Down Expand Up @@ -60,6 +73,22 @@ def process_text(text_input: TextInput) -> Any:
result = server.serve_text(text_input.text)
return {"message": result, "status": "ok"}

@app.post("/process_file_and_data/")
async def process_file_and_data(
file: UploadFile = File(...),
complex_data: Json[ComplexItem] = Form(...)
) -> Dict[str, Any]:
# You can now access the file and the complex data
# For example, save the file and process the complex data
# Here, just return a message indicating success for demonstration

return {
"message": "Received file and complex data successfully!",
"file_name": file.filename,
"complex_data_id": complex_data.id,
"complex_data_item_name": complex_data.item.name
}

# This port number must match the port number in the Dockerfile
if __name__ == "__main__":
uvicorn.run(app, host="0.0.0.0", port=80)

0 comments on commit ffdcdfd

Please sign in to comment.