RC: RDI In the Cloud

content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/_index.md

@@ -0,0 +1,22 @@
+---
+Title: Prepare AWS RDS and Aurora databases for RDI
+aliases: /integrate/redis-data-integration/ingest/data-pipelines/prepare-dbs/aws-aurora-rds/
+alwaysopen: false
+categories:
+- docs
+- integrate
+- rs
+- rdi
+description: Learn how to prepare AWS RDS and Aurora databases for RDI.
+group: di
+linkTitle: Prepare AWS RDS and Aurora
+summary: Prepare AWS Aurora and AWS RDS databases to work with Redis Data Integration.
+hideListLinks: false
+type: integration
+weight: 5
+---
+
+You can use RDI with databases on [AWS Relational Database Service (RDS)](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Welcome.html) and [AWS Aurora](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/CHAP_AuroraOverview.html).
+
+The pages in this section give detailed instructions to get your source
+database ready for Debezium to use:

content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-mysql.md

@@ -0,0 +1,78 @@
+---
+Title: Prepare AWS Aurora MySQL/AWS RDS MySQL for RDI
+aliases: /integrate/redis-data-integration/ingest/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-mysql/
+alwaysopen: false
+categories:
+- docs
+- integrate
+- rs
+- rdi
+description: Enable CDC features in your source databases
+group: di
+hideListLinks: false
+linkTitle: Prepare AWS Aurora/RDS MySQL
+summary: Prepare AWS Aurora MySQL and AWS RDS MySQL databases to work with Redis Data Integration.
+type: integration
+weight: 2
+---
+
+Follow the steps in the sections below to prepare an [AWS Aurora MySQL](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/CHAP_GettingStartedAurora.CreatingConnecting.Aurora.html) or [AWS RDS MySQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.CreatingConnecting.MySQL.html) database.
+database to work with RDI.
+
+## Create and apply parameter group
+
+RDI requires some changes to database parameters. On AWS RDS and AWS Aurora, you change these parameters via a parameter group.
+
+1. In the [Relational Database Service (RDS) console](https://console.aws.amazon.com/rds/),navigate to **Parameter groups > Create parameter group**. [Create a parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithParamGroups.CreatingCluster.html) with the following settings:
+
+    | Name | Value |
+    | :-- | :-- |
+    | **Parameter group name**  | Enter a suitable parameter group name, like `rdi-mysql` |
+    | **Description**  | (Optional) Enter a description for the parameter group |
+    | **Engine Type**  | Choose **Aurora MySQL** for Aurora MySQL or **MySQL Community** for AWS RDS MySQL.  |
+    | **Parameter group family**  | Choose **aurora-mysql8.0** for Aurora MySQL or **mysql8.0** for AWS RDS MySQL. |
+
+    Select **Create** to create the parameter group.
+
+1. Navigate to **Parameter groups** in the console. Select the parameter group you have just created and then select **Edit**. Change the following parameters:
+
+    | Name | Value |
+    | :-- | :-- |
+    | `binlog_format`  | `ROW` |
+    | `binlog_row_image`  | `FULL` |
+
+    Select **Save Changes** to apply the changes to the parameter group.
+
+1. Go back to your target database on the RDS console, select **Modify** and then scroll down to **Additional Configuration**. Set the **DB Cluster Parameter Group** to the group you just created. 
+
+    Select **Save changes** to apply the parameter group to the new database.
+
+## Create Debezium user
+
+The Debezium connector needs a user account to connect to MySQL. This
+user must have appropriate permissions on all databases where you want Debezium
+to capture changes.
+
+1. Connect to your database as an admin user and create a new user for the connector:
+
+    ```sql
+    CREATE USER '<username>'@'%' IDENTIFIED BY '<password>';
+    ```
+
+    Replace `<username>` and `<password>` with a username and password for the new user.
+
+    The `%` means that the user can connect from any client. If you want to restrict the user to connect only from the RDI host, replace `%` with the IP address of the RDI host.
+
+1. Grant the user the necessary permissions:
+
+    ```sql
+    GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT, LOCK TABLES ON *.* TO '<username>'@'%';
+    ```
+
+    Replace `<username>` with the username of the Debezium user.
+
+1. Finalize the user's permissions:
+
+    ```sql
+    FLUSH PRIVILEGES;
+    ```

content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md

@@ -0,0 +1,82 @@
+---
+Title: Prepare AWS Aurora PostgreSQL/AWS RDS PostgreSQL for RDI
+aliases: 
+- /integrate/redis-data-integration/ingest/data-pipelines/prepare-dbs/aws-aur-pgsql/
+- /integrate/redis-data-integration/ingest/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql/
+- /integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aur-pgsql/
+alwaysopen: false
+categories:
+- docs
+- integrate
+- rs
+- rc
+- rdi
+description: Prepare AWS Aurora PostgreSQL databases to work with RDI
+group: di
+linkTitle: Prepare AWS Aurora PostgreSQL
+summary: Prepare AWS Aurora PostgreSQL databases to work with Redis Data Integration.
+type: integration
+weight: 1
+---
+
+Follow the steps in the sections below to prepare an
+[AWS Aurora PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/CHAP_GettingStartedAurora.CreatingConnecting.AuroraPostgreSQL.html) or [AWS RDS PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.CreatingConnecting.PostgreSQL.html)
+database to work with RDI.
+
+## Create and apply parameter group
+
+RDI requires some changes to database parameters. On AWS RDS and AWS Aurora, you change these parameters via a parameter group.
+
+1. In the [Relational Database Service (RDS) console](https://console.aws.amazon.com/rds/), navigate to **Parameter groups > Create parameter group**. [Create a parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_WorkingWithParamGroups.CreatingCluster.html) with the following settings:
+
+    | Name | Value |
+    | :-- | :-- |
+    | **Parameter group name**  | Enter a suitable parameter group name, like `rdi-aurora-pg` or `rdi-rds-pg` |
+    | **Description**  | (Optional) Enter a description for the parameter group |
+    | **Engine Type**  | Choose **Aurora PostgreSQL** for Aurora PostgreSQL or **PostgreSQL** for AWS RDS PostgreSQL. |
+    | **Parameter group family**  | Choose **aurora-postgresql15** for Aurora PostgreSQL or **postgresql13** for AWS RDS PostgreSQL. |
+
+    Select **Create** to create the parameter group.
+
+1. Navigate to **Parameter groups** in the console. Select the group you have just created and then select **Edit**. Change the following parameters:
+
+    | Name | Value |
+    | :-- | :-- |
+    | `rds.logical_replication`  | `1` |
+
+    Select **Save Changes** to apply the changes to the parameter group.
+
+1. Go back to your database on the RDS console, select **Modify** and then scroll down to **Additional Configuration**. Set the **DB Cluster Parameter Group** to the group you just created. 
+
+    Select **Save changes** to apply the parameter group to your database.
+
+## Create Debezium user
+
+The Debezium connector needs a user account to connect to PostgreSQL. This
+user must have appropriate permissions on all databases where you want Debezium
+to capture changes.
+
+1. Connect to PostgreSQL as the `postgres` user and create a new user for the connector:
+
+    ```sql
+    CREATE ROLE <username> WITH LOGIN PASSWORD '<password>' VALID UNTIL 'infinity';
+    ```
+
+    Replace `<username>` and `<password>` with a username and password for the new user.
+
+1. Grant the user the necessary replication permissions:
+
+    ```sql
+    GRANT rds_replication TO <username>;
+    ```
+
+    Replace `<username>` with the username of the Debezium user.
+
+1. Connect to your database as the `postgres` user and grant the new user access to one or more schemas in the database:
+
+    ```sql
+    GRANT SELECT ON ALL TABLES IN SCHEMA <schema> TO <username>;
+    ```
+
+    Replace `<username>` with the username of the Debezium user and `<schema>` with the schema name.
+

content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-rds-sqlserver.md

@@ -0,0 +1,100 @@
+---
+Title: Prepare Microsoft SQL Server on AWS RDS for RDI
+aliases: /integrate/redis-data-integration/ingest/data-pipelines/prepare-dbs/aws-aurora-rds/aws-rds-sqlserver/
+alwaysopen: false
+categories:
+- docs
+- integrate
+- rs
+- rdi
+description: Enable CDC features in your source databases
+group: di
+hideListLinks: false
+linkTitle: Prepare Microsoft SQL Server on AWS RDS
+summary: Prepare Microsoft SQL Server on AWS RDS databases to work with Redis Data Integration.
+type: integration
+weight: 3
+---
+
+Follow the steps in the sections below to prepare a [Microsoft SQL Server on AWS RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.CreatingConnecting.SQLServer.html) database to work with RDI.
+
+## Create the Debezium user
+
+The Debezium connector needs a user account to connect to SQL Server. This
+user must have appropriate permissions on all databases where you want Debezium
+to capture changes.
+
+1. Connect to your database as an admin user and create a new user for the connector:
+
+    ```sql
+    USE master
+    GO
+    CREATE LOGIN <username> WITH PASSWORD = '<password>'
+    GO
+    USE <database>
+    GO
+    CREATE USER <username> FOR LOGIN <username>
+    GO
+    ```
+
+    Replace `<username>` and `<password>` with a username and password for the new user and replace `<database>` with the name of your database.
+
+1. Grant the user the necessary permissions:
+
+    ```sql
+    USE master
+    GO
+    GRANT VIEW SERVER STATE TO <username>
+    GO
+    USE <database>
+    GO
+    EXEC sp_addrolemember N'db_datareader', N'<username>'
+    GO
+    ```
+
+    Replace `<username>` with the username of the Debezium user and replace `<database>` with the name of your database.
+
+## Enable CDC on the database
+
+Change Data Capture (CDC) must be enabled for the database and for each table you want to capture.
+
+1. Enable CDC for the database by running the following command:
+
+    ```sql
+    EXEC msdb.dbo.rds_cdc_enable_db '<database>'
+    GO
+    ```
+
+    Replace `<database>` with the name of your database.
+
+1. Enable CDC for each table you want to capture by running the following commands:
+
+    ```sql
+    USE <database>
+    GO
+    EXEC sys.sp_cdc_enable_table
+    @source_schema = N'<schema>',
+    @source_name   = N'<table>', 
+    @role_name     = N'<role>',
+    @supports_net_changes = 0
+    GO
+    ```
+
+    Replace `<database>` with the name of your database, `<schema>` with the name of the schema containing the table, `<table>` with the name of the table, and `<role>` with the name of a new role that will be created to manage access to the CDC data. 
+
+    {{< note >}}
+The value for `@role_name` can’t be a fixed database role, such as `db_datareader`. 
+Specifying a new name will create a corresponding database role that has full access to the
+captured change data.
+    {{< /note >}}
+
+1. Add the Debezium user to the CDC role:
+
+    ```sql
+    USE <database>
+    GO
+    EXEC sp_addrolemember N'<role>', N'<username>'
+    GO
+    ```
+
+    Replace `<role>` with the name of the role you created in the previous step and replace `<username>` with the username of the Debezium user.