Skip to content

feat(cloud): add cloud-run guide #356

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions docs/cloud/connect/serverless/aws-lambda.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,24 +124,24 @@ export const handler = async (event) => {

---

### 6. Connecting to a Private Dragonfly Data Store
## Connecting to a Private Dragonfly Data Store

Private data stores are hosted within a Virtual Private Cloud (VPC), which provides an isolated network environment. To enable your AWS Lambda function to securely connect to a private Dragonfly data store, follow these beginner-friendly steps:

#### 1. Set Up VPC Peering
### 1. Set Up VPC Peering

1. Create a VPC in your AWS account within the same region as your data store.
2. Establish a peering connection between your VPC and the data store's VPC. This allows the two networks to communicate. For detailed guidance, refer to the [VPC Peering Connections documentation](../../connections.md).

#### 2. Adjust Security Group Rules
### 2. Adjust Security Group Rules

1. Open the [VPC Console](https://console.aws.amazon.com/vpc/) and locate the security group associated with your vpc.
2. Add an inbound rule to allow traffic from your vpc:
- **Type**: Custom TCP Rule
- **Port Range**: `6379` (Dragonfly port).
- **Source**: CIDR of the private network.

#### 3. Grant Lambda the Necessary Permissions
### 3. Grant Lambda the Necessary Permissions

To allow Lambda to interact with your VPC, you need to update its execution role:

Expand All @@ -150,7 +150,7 @@ To allow Lambda to interact with your VPC, you need to update its execution role
3. Under **Permissions**, click the execution role name.
4. Add the **AmazonEC2FullAccess** permission to the role. This ensures Lambda can connect to your VPC.

#### 4. Configure Lambda to Use the VPC
### 4. Configure Lambda to Use the VPC

1. In the [AWS Lambda Console](https://console.aws.amazon.com/lambda/), select your function.
2. Go to the **Configuration** tab and choose **VPC**.
Expand All @@ -159,7 +159,7 @@ To allow Lambda to interact with your VPC, you need to update its execution role
- **Subnets**: Choose subnets with access to the data store.
- **Security Groups**: Select the security group that allows traffic to the data store.

#### 5. Test the Connection
### 5. Test the Connection

1. Deploy your Lambda function as described earlier.
2. Update the `DRAGONFLY_CONNECTION_URI` environment variable with the private data store's connection URL.
Expand Down
180 changes: 180 additions & 0 deletions docs/cloud/connect/serverless/cloud-run.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
---
sidebar_position: 2
---

# Cloud Run
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets title as GCP Cloud Run


Cloud Run is a serverless compute service provided by GCP. It allows you to run code without provisioning or managing servers. It supports various programming languages and integrates seamlessly with other GCP services, making it ideal for building scalable, event-driven applications.

This guide explains how to connect a Cloud run service to a Dragonfly Cloud instance.

---

## Prerequisites

1. **Dragonfly Cloud Instance**: Ensure you have a running Dragonfly Cloud instance and its connection URI.
2. **GCP Console**: Access to Cloud Run and IAM services.
3. **Go Runtime**: The Cloud Run service will be written in NodeJS.
4. **Redis Client Library**: Use the `go-redis/v9` package to interact with Dragonfly.

---

## Cloud Run function code

I am writing a function for this guide for simplicity. You can deploy a service
instead. The process to connect to a dragonfly data store is same. I will deploy
the below sample code -

```go
package helloworld

import (
"encoding/json"
"fmt"
"html"
"net/http"
"os"
"context"

"github.com/redis/go-redis/v9"

"github.com/GoogleCloudPlatform/functions-framework-go/functions"

)

var addr string
var pass string

func init() {
functions.HTTP("HelloHTTP", helloHTTP)
}

// helloHTTP is an HTTP Cloud Function with a request parameter.
func helloHTTP(w http.ResponseWriter, r *http.Request) {
var d struct {
Name string `json:"name"`
}
if err := json.NewDecoder(r.Body).Decode(&d); err != nil {
fmt.Fprint(w, "Hello, World!")
return
}

setDragonflyValue(d.Name)
if d.Name == "" {
fmt.Fprint(w, "Hello, World!")
return
}
fmt.Fprintf(w, "Hello, %s!", html.EscapeString(d.Name))
}

func setDragonflyValue(name string) {
ctx := context.Background()
addr = os.Getenv("DFADDR") // format- <datastore-host>:<port>
pass = os.Getenv("DFPASS") // datastore password

// Create a Redis client
client := redis.NewClient(&redis.Options{
Addr: addr,
Password: pass,
DB: 0, // Use default DB
})

// Ping the server to test the connection
pong, err := client.Ping(ctx).Result()
if err != nil {
fmt.Printf("Error connecting to Dragonfly: %v\n", err)
return
}
fmt.Printf("Connected to Dragonfly: %s\n", pong)

// Perform some test operations
err = client.Set(ctx, "test_key", name, 0).Err()
if err != nil {
fmt.Printf("Error setting key: %v\n", err)
return
}

value, err := client.Get(ctx, "test_key").Result()
if err != nil {
fmt.Printf("Error getting key: %v\n", err)
return
}
fmt.Printf("Retrieved value: %s\n", value)

// Close the connection
err = client.Close()
if err != nil {
fmt.Printf("Error closing connection: %v\n", err)
}
}
```

The code uses two environment variables `DFADDR` and `DFPASS`.

## Steps to Connect from the Cloud Run Function to a public data store

### 1. Create a New Cloud Run Function

1. Go to the [Cloud Run](https://console.cloud.google.com/run).
2. Click **Write a function**.
3. Choose **Go Runtime**.
4. Provide a name for your function (e.g., `DragonflyConnector`).
5. Expand the Containers section.
6. Edit the **Container Port** to match with the data store's port.
7. Add `DFADDR` and `DFPASS` environment variables.
8. Click **Create**.

### 2. Test the Cloud Run Function

1. Click the **Test** button in the console.
2. Create a new test event (you can use the default template).
3. Run the test in cloud shell.
4. Check the logs in **Logs** to verify the connection and the key-value pair operation.

---

## Connect to a Private Datastore

Private datastores are hosted within a Virtual Private Cloud (VPC), which provides
an isolated network environment. To enable your Cloud Run function to securely
connect to a private Dragonfly datastore, follow these beginner-friendly steps:

### 1. Set Up VPC Peering

1. Create a VPC in your GCP account within the same region as your datastore.
2. Establish a peering connection between your VPC and the datastore's VPC. This allows the two networks to communicate. For detailed guidance, refer to the [VPC Peering Connections documentation](../../connections.md).

### 2. Adjust Firewall Rules

1. Open the [VPC Network Console](https://console.cloud.google.com/networking/networks/list).
2. Select your network and open the Firewall settings.
3. Add an ingress rule to allow traffic from your datastore vpc. Put datastore vpc CIDR range in the Source Ipv4 range field. Allow all ports.

### 3. Edit Cloud Run settings

As the data store is private, you need to configure your cloud run service's network setting to the
VPC network you just created.

1. Go to the [Cloud Run](https://console.cloud.google.com/run). Select your service.
2. Navigate to the **Networking** tab.
3. Select **Internal Ingress**. Save changes.
4. Once deployed, click **Edit & deploy new version**.
5. Edit container port to your datastore's port. Update `DFADDR` and `DFPASS` (empty if passkey is not set).
6. Go to **Networking**. Select **Connect to a VPC for outbound traffic**. Choose your vpc.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is probably the most important part but it is buried in a lot of details about setting up the cloud run function.

I think we should start with some minimal clear steps that assumes you have a coloud run service and a private endpoint data store

Then all that is needed is:

  1. Make sure dragonfly vpc is peer connected to the vpc cloud run is deployed too
  2. The Connect to a VPC for outbound traffic. Choose your vpc.setp
    ?

You can have a collapsable "end to end" example below if you want (or do it in a separate blog post which we can link form here)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, sure makes sense.

7. Deploy the changes.

### 4. Test the Connection

1. Click **Test**. Copy the test command.
2. You need to create a vm instance inside your vpc to run the test. Go to **VM instance** tab and
create a vm instance. Make sure you've configured the network interface to use your vpc.
3. Update your firewall rule so that you can connect to the instance via ssh.
4. SSH to your machine. Run the test command.

You'll see the logs in **Logs** that dragonfly has stored the value.

By following these steps, you can securely connect your Cloud run service to a private Dragonfly datastore, ensuring your application remains both scalable and secure.

## Conclusion

You have successfully created a GCP Cloud Run function that connects to Dragonfly Cloud, sets a test key-value pair, and verifies the connection. You can now extend this function to perform more complex operations with Dragonfly.
1 change: 1 addition & 0 deletions docs/cloud/connect/serverless/serverless.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@ slug: /cloud/connect/serverless
Dragonfly Cloud offers seamless integration with serverless services, enabling you to leverage its high-performance in-memory data store for your applications. This guide will walk you through the steps to connect your serverless environment to Dragonfly Cloud, ensuring optimal performance and compatibility. Whether you're using AWS Lambda, Google Cloud Functions, or Azure Functions, Dragonfly Cloud provides the tools and support you need to get started quickly and efficiently.

- [AWS Lambda](aws-lambda.md)
- [GCP Cloud Run](cloud-run.md)