Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add API Gateway Local setup #200

Merged
merged 3 commits into from
Dec 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ Hello there!

So, you want to help improve the site — great!

### Setup

Local setup is fairly straightforward:

1. Run the server (you'll need [Rust](https://www.rust-lang.org/)):
Expand Down Expand Up @@ -31,6 +33,8 @@ It will also auto-generate user votes over time for the questions there.
If you're curious about the technologies used in the server and client,
see their respective `README.md` files.

### DynamoDB Local

To run tests against a DynamoDB instance running [locally](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html), make sure
you got [`docker`](https://docs.docker.com/engine/install/) and
[`AWS CLI`](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html#getting-started-install-instructions) installed, then hit:
Expand All @@ -55,3 +59,49 @@ your local DynamoDB instance, hit:
```sh
USE_DYNAMODB=local cargo run
```

### API Gateway Local

Prerequisites:
jonhoo marked this conversation as resolved.
Show resolved Hide resolved

- [Cargo Lambda](https://www.cargo-lambda.info/guide/installation.html#binary-releases)
- [SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html)
- DynamoDB Local [container](#backend-with-dynamodb-local)

NB! `API Gateway Local` will only work when the binary is built in release mode (`sam build` will do this for us).
See how we are wrapping the `axum` app in the `LambdaLayer` in release mode in [main](./server/src/main.rs).

To build and launch the application as a `Lambda` function behind `API Gateway` locally, `cd` to the server
directory, and hit:

```sh
sam build
sam local start-api
```

Once you make changes to the back-end code, open a separate terminal window and rebuild the app with:

```sh
sam build
```

The `sam local` process we've lauched previously will then pick up the new binary from `./server/.aws-sam` directory.

Here is how our `API Gateway Local` plus `DynamoDB Local` setup look like:

```sh
______________________________ _______________________________________________
| Browser | | Docker Network: wewerewondering |
| _______________________ | _______________________ | __________________________________ |
| | | | | API Gateway Proxy | | | WeWereWondering Server Container | |
| | WeWereWodering Client |-- |--> | http://localhost:3000 | --|--> | ports: SAM assigns dynamically | --| |
| | http://localhost:5173 | | |_______________________| | |__________________________________| | |
| |_______________________| | | | |
| _______________________ | | | |
| | | | | _____________________________ | |
| | DynamoDB Admin Client |---|--------------------------------|--> | DynamoDB Local Container | | |
| | http://localhost:8001 | | | | ports: 127.0.0.1:8000:8000 | | |
| |_______________________| | | | host: dynamodb-local | <------| |
| | | |_____________________________| |
|______________________________| |_______________________________________________|
```
1 change: 1 addition & 0 deletions server/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@
/dynamodb-local
dynamodb_local_latest.tar.gz*
Makefile
.aws-sam
23 changes: 19 additions & 4 deletions server/run-dynamodb-local.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,13 @@ export AWS_ACCESS_KEY_ID=lorem
export AWS_SECRET_ACCESS_KEY=ipsum
export AWS_DEFAULT_REGION=us-east-1

DYNAMODB_NETWORK_NAME=wewerewondering
DYNAMODB_CONTAINER_NAME=dynamodb-local
DYNAMODB_ADMIN_CONTAINER_NAME=dynamodb-admin
DYNAMODB_HOST=127.0.0.1
DYNAMODB_PORT=8000
DYNAMODB_ADMIN_HOST=127.0.0.1
DYNAMODB_ADMIN_PORT=8001
ENDPOINT_URL=http://${DYNAMODB_HOST}:${DYNAMODB_PORT}

docker ps | grep ${DYNAMODB_CONTAINER_NAME} >/dev/null &&
Expand All @@ -34,11 +37,17 @@ echo "🖴 Preparing volumes for DynamoDB..."
rm -rf dynamodb-data
mkdir dynamodb-data

if docker network inspect ${DYNAMODB_NETWORK_NAME} 2>&1 >/dev/null; then
echo "🚫 Network ${DYNAMODB_NETWORK_NAME} already exists, re-using..."
else
docker network create ${DYNAMODB_NETWORK_NAME}
fi

echo "🚀 Spinning up a container with DynamoDB..."
(
docker run --rm -d -v ./dynamodb-data:/home/dynamodblocal/data -p ${DYNAMODB_HOST}:${DYNAMODB_PORT}:8000 \
-w /home/dynamodblocal --name ${DYNAMODB_CONTAINER_NAME} amazon/dynamodb-local:latest \
-jar DynamoDBLocal.jar -sharedDb -dbPath ./data
-w /home/dynamodblocal --name ${DYNAMODB_CONTAINER_NAME} --network ${DYNAMODB_NETWORK_NAME} \
amazon/dynamodb-local:latest -jar DynamoDBLocal.jar -sharedDb -dbPath ./data
) >/dev/null

while ! (aws dynamodb list-tables --endpoint-url ${ENDPOINT_URL} >/dev/null); do
Expand All @@ -54,7 +63,13 @@ docker ps | grep ${DYNAMODB_ADMIN_CONTAINER_NAME} >/dev/null &&
exit 0

echo "🚀 Spinning up a container with DynamoDB Admin..."
(docker run -d --rm --net host --name ${DYNAMODB_ADMIN_CONTAINER_NAME} aaronshaf/dynamodb-admin) >/dev/null
echo "🔎 DynamoDB Admin is available at http://localhost:8001"
(
docker run -d --rm -p ${DYNAMODB_ADMIN_HOST}:${DYNAMODB_ADMIN_PORT}:8001 \
--name ${DYNAMODB_ADMIN_CONTAINER_NAME} \
--network ${DYNAMODB_NETWORK_NAME} \
-e DYNAMO_ENDPOINT=http://${DYNAMODB_CONTAINER_NAME}:8000 \
aaronshaf/dynamodb-admin
) >/dev/null
echo "🔎 DynamoDB Admin is available at http://${DYNAMODB_ADMIN_HOST}:${DYNAMODB_ADMIN_PORT}"

echo "✅ Done!"
40 changes: 40 additions & 0 deletions server/samconfig.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# NB! We are only using SAM to run the application as a Lambda function behind
# the API Gateway locally. We are actually using Terraform to describe and deploy
# the infrastructure (see `infra` directory in the project's root with IaC files).

# More information about the configuration file can be found here:
# https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html
version = 0.1
stack_name = "wewerewondering"

[default.build.parameters]
# Cargo Lambda is supported as a beta feature in SAM:
# https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/building-rust.html
beta_features = true
cached = true
parallel = true

[default.validate.parameters]
lint = true

[default.deploy.parameters]
capabilities = "CAPABILITY_IAM"
confirm_changeset = true
resolve_s3 = true

[default.package.parameters]
resolve_s3 = true

[default.sync.parameters]
beta_features = true
watch = true

[default.local_start_api.parameters]
warm_containers = "EAGER"
# The command `sam local start-api` will be launch Lambda Runtime in a docker
# container and so we need to make sure the DynamoDB Local is in the same network
# (see `DYNAMODB_NETWORK_NAME` in `./run-dynamodb-local.sh`)
docker_network = "wewerewondering"

[default.local_start_lambda.parameters]
warm_containers = "EAGER"
33 changes: 25 additions & 8 deletions server/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,10 @@ impl Backend {

/// Instantiate a DynamoDB backend.
///
/// If `USE_DYNAMODB` is set to "local", the `AWS_ENDPOINT_URL` will be set
/// to "http://localhost:8000", the `AWS_DEFAULT_REGION` will be "us-east-1",
/// and the [test credentials](https://docs.rs/aws-config/latest/aws_config/struct.ConfigLoader.html#method.test_credentials)
/// If `USE_DYNAMODB` is set to "local", the `AWS_ENDPOINT_URL` will be taken
/// from the environment with the "http://localhost:8000" fallback , the `AWS_DEFAULT_REGION`
/// will be pulled from the environment as well and will default to "us-east-1",
/// as for the credentials - the [test credentials](https://docs.rs/aws-config/latest/aws_config/struct.ConfigLoader.html#method.test_credentials)
/// will be used to sign requests.
///
/// This spares setting those environment variables (including `AWS_ACCESS_KEY_ID`
Expand All @@ -56,17 +57,33 @@ impl Backend {
/// USE_DYNAMODB=local cargo t -- --include-ignored
/// ```
///
/// If customization is needed, set `USE_DYNAMODB` to e.g. "custom", and
/// set the evironment variables to whatever values you need or let them be
/// picked up from your `~/.aws` files (see [`aws_config::load_from_env`](https://docs.rs/aws-config/latest/aws_config/fn.load_from_env.html))
/// This also allows us to use the local instance of DynamoDB which is running
/// in a container on the same network, in which case the database will be accessible
/// under `http://<dynamodb_container_name>:<port>`. This facilitates the setup of
/// local API Gateway with SAM, since the `sam local start-api` command will launch our
/// back-end app in a docker container.
///
/// If more customization is needed (say, you want to set some specific credentials
/// rather than rely on those test creds generated by the `aws_config` crate),
/// set `USE_DYNAMODB` to e.g. "custom", and set the environment variables to whatever
/// values you need or let them be picked up from your `~/.aws` files
/// (see [`aws_config::load_from_env`](https://docs.rs/aws-config/latest/aws_config/fn.load_from_env.html))
async fn dynamo() -> Self {
let config = if std::env::var("USE_DYNAMODB")
.ok()
.is_some_and(|v| v == "local")
{
aws_config::from_env()
.endpoint_url("http://localhost:8000")
.region("us-east-1")
.endpoint_url(
std::env::var("AWS_ENDPOINT_URL")
.ok()
.unwrap_or("http://localhost:8000".into()),
)
.region(aws_config::Region::new(
std::env::var("AWS_DEFAULT_REGION")
.ok()
.unwrap_or("us-east-1".into()),
))
.test_credentials()
.load()
.await
Expand Down
67 changes: 67 additions & 0 deletions server/template.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# NB! We are only using SAM to run the application as a Lambda function behind
# the API Gateway locally. We are actually using Terraform to describe and deploy
# the infrastructure (see `infra` directory in the project's root with IaC files,
# specifically `infra/apigw.tf` and `infra/lambda.tf`).

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Sample SAM Template for running and tesing WeWereWondering locally
Resources:
WeWereWonderingApi:
Type: AWS::Serverless::Function
Metadata:
BuildMethod: rust-cargolambda
Properties:
CodeUri: .
Handler: bootstrap
Runtime: provided.al2
Architectures:
- x86_64
Timeout: 29
MemorySize: 512
Events:
FetchEvent:
Type: HttpApi
Properties:
Path: /api/event/{eid}
Method: get
CreateEvent:
Type: HttpApi
Properties:
Path: /api/event
Method: post
AskQuestion:
Type: HttpApi
Properties:
Path: /api/event/{eid}
Method: post
FetchAllUnhiddenQuestionsForEvent:
Type: HttpApi
Properties:
Path: /api/event/{eid}/questions
Method: get
FetchFetchAllQuestionsForEventAllQuestionsForEvent:
Type: HttpApi
Properties:
Path: /api/event/{eid}/questions/{secret}
Method: get
ToggleQuestionProperty:
Type: HttpApi
Properties:
Path: /api/event/{eid}/questions/{secret}/{qid}/toggle/{property}
Method: post
UpvoteDownvoteQuestion:
Type: HttpApi
Properties:
Path: /api/vote/{qid}/{updown}
Method: post
FetchQuestions:
Type: HttpApi
Properties:
Path: /api/questions/{qids}
Method: get
Environment:
Variables:
RUST_LOG: debug
USE_DYNAMODB: local
AWS_ENDPOINT_URL: http://dynamodb-local:8000
Loading