diff --git a/content/en/experiments/connecting_redshift.md b/content/en/experiments/connecting_redshift.md new file mode 100644 index 00000000000..e3aed2d3af5 --- /dev/null +++ b/content/en/experiments/connecting_redshift.md @@ -0,0 +1,165 @@ +--- +title: Connect Redshift for Warehouse Native Experiment Analysis +description: Connect a Redshift service account to enable warehouse native experiment analysis. +private: true +further_reading: +- link: "/experiments/defining_metrics" + tag: "Documentation" + text: "Defining metrics in Datadog Experiments" +- link: "https://www.datadoghq.com/blog/experimental-data-datadog/" + tag: "Blog" + text: "How to bridge speed and quality in experiments through unified data" +--- + +## Overview + +This guide walks through connecting AWS Redshift to Datadog to enable warehouse-native experiment analysis in four steps: connecting AWS to Datadog, creating necessary resources in AWS, granting permissions to the Datadog service user, and configuring experiment-specific settings in Datadog. + +## Step 1: Connect AWS to Datadog + +Datadog Experiments connects to Redshift through the Datadog AWS integration. If you already have an [AWS integration][1] configured for the account containing your Redshift cluster, you can skip to the next step. + +If you haven't set up the AWS integration yet: + +1. Navigate to the [AWS integration tile][2] in Datadog. +2. Click **Add AWS Account** and follow the setup flow using the CloudFormation template. This creates an IAM role that allows Datadog to make API calls to your AWS account. +3. For more details, see the [AWS integration documentation][1]. + +If you're only using the AWS integration for warehouse native experiment analysis, you can opt out of collecting other resources. + +## Step 2: Create resources in AWS + +Before you connect Datadog Experiments to your data warehouse, create a service user for Datadog. + +You will also need to create a schema for Datadog Experiments to write intermediary tables to, as well as grant the service User read access to tables you'd like Datadog Experiments to query SQL definitions from. + +### Create a Service User for Datadog Experiments + +1. Connect to your Redshift cluster with a user that has superuser or admin privileges. +2. Create a user with the following command: + +```sql +CREATE USER datadog_experiments_user PASSWORD ''; +``` + +3. Grant this user the appropriate privileges for any tables needed for building experiment metrics. Replace `` and `` with the appropriate values. + +```sql +GRANT USAGE ON SCHEMA TO datadog_experiments_user; + +-- individual tables +GRANT SELECT ON TABLE .
TO datadog_experiments_user; + +-- all tables +GRANT SELECT ON ALL TABLES IN SCHEMA TO datadog_experiments_user; +``` + +### Create Schema for Datadog Experiments to Write Results to + +Create a schema for Datadog Experiments to write intermediate results and temporary tables. + +```sql +CREATE SCHEMA IF NOT EXISTS datadog_experiments_output; +GRANT ALL ON SCHEMA datadog_experiments_output TO datadog_experiments_user; +``` + +### Create an S3 Bucket + +Create an S3 bucket for importing exposure events into your warehouse. The bucket must start with `"datadog-experimentation-"`, e.g. `datadog-experimentation-[AWS account ID]`. Default settings can be used. + +## Step 3: Grant additional IAM permissions + +Because Datadog Experiments writes data into your warehouse, your AWS IAM role used by the Datadog integration needs additional permissions beyond those required for the standard [Amazon Redshift Datadog integration][1]. + +Add the following permissions to the IAM role used by your Datadog AWS integration: + +```JSON +{ + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "GetClusterCreds", + "Effect": "Allow", + "Action": [ + "redshift:GetClusterCredentials" + ], + "Resource": [ + "[Redshift cluster ARN]", + "[Redshift user ARN]", + "[Redshift database ARN]" + ] + }, + { + "Sid": "QueryRedshift", + "Effect": "Allow", + "Action": [ + "redshift-data:ExecuteStatement", + "redshift-data:GetStatementResult", + "redshift-data:DescribeStatement", + "redshift-data:ListStatements", + "redshift-data:CancelStatement" + ], + "Resource": "*" + }, + { + "Sid": "ListTheBucket", + "Effect": "Allow", + "Action": [ + "s3:ListBucket" + ], + "Resource": "[S3 bucket ARN]" + }, + { + "Sid": "ObjectRW", + "Effect": "Allow", + "Action": [ + "s3:GetObject", + "s3:PutObject", + "s3:DeleteObject" + ], + "Resource": "[S3 bucket ARN]/*" + } + ] +} +``` + +Replace the following with the appropriate values: + +| Field | Example | +|-------|---------| +| `[Redshift cluster ARN]` | `arn:aws:redshift:us-east-1:[account-id]:namespace:[namespace-id]` | +| `[Redshift user ARN]` | `arn:aws:redshift:us-east-1:312078929830:dbuser:[cluster-name]/[user]` | +| `[Redshift database ARN]`| `arn:aws:redshift:us-east-1:[account-id]:dbname:[cluster-name]` | +| `[S3 bucket ARN]` | `arn:aws:s3:::[bucket-name]` | + +## Step 4: Configure experiment settings + +Once your AWS service user is connected to Datadog, follow these steps to finish the integration. + +1. Navigate to the [Warehouse Connections page][3]. +2. Click **Redshift**. +3. Under **Select AWS Account**, choose the AWS account you configured above. +4. Under **Cluster Connection**, enter: + - **AWS region** — the region your Redshift cluster is in (e.g., `us-east-1`) + - **Cluster identifier** — the name of your Redshift cluster + - **Cluster endpoint** — the full endpoint URL for your cluster + - **Port** — the port your cluster is listening on (default: `5439`) +5. Under Database and Storage, enter: + - **Database** — the database containing your source tables + - **Database user** — the service user you created earlier (e.g., `datadog_experiments_user`) + - **Schema** — the schema you created for Datadog Experiments to write to (e.g., `datadog_experiments_output`) + - **Temp S3 bucket** — the S3 bucket you created earlier (e.g., `datadog-experimentation-[AWS account ID]`) +6. Click **Save** + +{{< img src="/product_analytics/experiment/guide/redshift_experiment_setup.png" alt="The Edit Data Warehouse modal with Redshift selected, showing inputs for AWS account, AWS region, cluster identifier, cluster endpoint ,database, database user, schema, and temp S3 bucket" style="width:90%;" >}} + +After you save your warehouse connection, create experiment metrics using your Redshift data. See [Create Experiment Metrics][4]. + +## Further reading + +{{< partial name="whats-next/whats-next.html" >}} + +[1]: https://docs.datadoghq.com/integrations/amazon-web-services/ +[2]: https://app.datadoghq.com/integrations/aws +[3]: https://app.datadoghq.com/product-analytics/experiments/settings/warehouse-connections +[4]: /experiments/defining_metrics \ No newline at end of file diff --git a/static/images/product_analytics/experiment/guide/redshift_experiment_setup.png b/static/images/product_analytics/experiment/guide/redshift_experiment_setup.png new file mode 100644 index 00000000000..c93e238d927 Binary files /dev/null and b/static/images/product_analytics/experiment/guide/redshift_experiment_setup.png differ