Skip to content

Commit

Permalink
added fastapi-server instrucs
Browse files Browse the repository at this point in the history
  • Loading branch information
pchalasani committed Feb 29, 2024
1 parent 24a1f1c commit 7b47bf0
Show file tree
Hide file tree
Showing 4 changed files with 52 additions and 19 deletions.
1 change: 1 addition & 0 deletions fastapi-server/.dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@
.venv
poetry.lock
pyproject.toml
README.md

2 changes: 1 addition & 1 deletion fastapi-server/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM --platform=linux/amd64 python:3.11
FROM python:3.11


# Set the working directory in the container
Expand Down
15 changes: 10 additions & 5 deletions fastapi-server/Makefile
Original file line number Diff line number Diff line change
@@ -1,14 +1,19 @@
LOCAL_ARCH=$(shell uname -m)
GCLOUD_ARCH=amd64
IMAGE_NAME=langroid-server
TAG=latest

server:
docker build -t server .
docker build --platform linux/$(LOCAL_ARCH) -t $(IMAGE_NAME):$(TAG) .

run:
docker run -d -p 80:80 server
docker run --env-file .env -d -p 80:80 $(IMAGE_NAME):$(TAG)

stop:
docker stop $(shell docker ps -q)

gbuild:
docker build -t gcr.io/langroid/langroid-server:v1 .
gserver:
docker build --platform=linux/$(GCLOUD_ARCH) -t gcr.io/langroid/$(IMAGE_NAME):$(TAG) .

gpush:
docker push gcr.io/langroid/langroid-server:v1
docker push gcr.io/langroid/$(IMAGE_NAME):$(TAG)
53 changes: 40 additions & 13 deletions fastapi-server/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,16 @@
# Setup a REST API server on GCP for a Langroid script

Ensure that your env secrets are in this folder as a `.env` file.
We will of course *not* include it in the docker image
(which is why it's in the `.dockerignore` file) but we will use it
for local testing by passing this file in as an env file to the uvicorn server.
(See the defn of `make run` in the Makefile.)

## local testing

Build server using `make server`, run it with `make run`
Build server using `make server`, run it with `make run`.
See the definitions of these in the Makefile. Notice that
we are passing in the env variables

## Curl examples to test the server locally

Expand Down Expand Up @@ -36,20 +44,12 @@ See other details here:
https://chat.openai.com/share/c34583c8-b88e-4a70-bf24-83229700c020


Run these from within the dir where the Dockerfile is located:

```bash
gcloud auth configure-docker
docker build -t gcr.io/langroid/langroid-server:v1 .
docker push gcr.io/langroid/langroid-server:v1
```

The `build` and `push` cmds are also available via the Makefile
as `make gbuild` and `make gpush` respectively.
Run `make gserver`, `make gpush` from within the dir where the Dockerfile is
located.

Go to Google Cloud Run Service and create a new service,
selecting the latest version of the pushed docker image above:
`gcr.io/langroid/langroid-server:v1`
selecting the latest version of the pushed docker image above, e.g.:
`gcr.io/langroid/langroid-server:latest`

When setting up the service:
- ensure you select the same port number as in the Dockerfile, e.g. 80.
Expand All @@ -61,6 +61,33 @@ If the service fails to start due to an error like `uvicorn: exec format error`,
then you may be able to fix it by explicitly choosing an architecture
in the Dockerfile, e.g. `linux/amd64` (which we chose in the Dockerfile).

### Creating secrets in google cloud
Some commands for quick reference:

```bash
gcloud secrets create openai-api-key --replication-policy="automatic"
echo -n "your-openai-api-key" | gcloud secrets versions add openai-api-key --data-file=-
```

After creating your secret and adding its value, you may need to set appropriate
permissions for the secret. Use gcloud secrets add-iam-policy-binding to grant access
to the secret:

```bash
gcloud secrets add-iam-policy-binding openai-api-key \
--member="serviceAccount:[email protected]" \
--role="roles/secretmanager.secretAccessor"
```

To expose one of these as environment var named `OPENAI_API_KEY` in the cloud run
service:

```bash
gcloud run services update langroid-server \
--update-secrets OPENAI_API_KEY=openai-api-key:latest \
--region=us-east4
```

## Test GCP endpoints

Same curl cmds as above, but use the endpoint url from the GCP Cloud Run service,
Expand Down

0 comments on commit 7b47bf0

Please sign in to comment.