Skip to content

Commit

Permalink
Second round of the initial PR change requests
Browse files Browse the repository at this point in the history
  • Loading branch information
bobbyiliev committed Oct 15, 2024
1 parent f06f9b0 commit 232787a
Show file tree
Hide file tree
Showing 5 changed files with 35 additions and 5 deletions.
17 changes: 16 additions & 1 deletion docs/guides/materialize_source_table.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,12 @@
---
# generated by https://github.com/hashicorp/terraform-plugin-docs
# template file: templates/guides/materialize_source_table.md.tmpl
page_title: "Source Table Migration Guide"
subcategory: ""
description: |-
Guide for migrating to the new materialize_source_table_{source_type} resources.
---

# Source versioning: migrating to `materialize_source_table_{source_type}` Resource

In previous versions of the Materialize Terraform provider, source tables were defined within the source resource itself and were considered subsources of the source rather than separate entities.
Expand Down Expand Up @@ -169,7 +178,7 @@ resource "materialize_source_kafka" "kafka_source" {

In the `lifecycle` block, add the `ignore_changes` meta-argument to prevent Terraform from trying to update these attributes during subsequent applies, that way Terraform won't try to update these values based on incomplete information from the state as they will no longer be defined in the source resource itself but in the new `materialize_source_table_{source_type}` resources.

> Note: We will make the changes to those attributes a no-op, so the `ignore_changes` block will not be necessary.
> Note: Once the migration process is fully implemented on the Materialize side and the attributes will have to be updated as no-op in future versions of the provider. That way the `ignore_changes` block will no longer be required. At that point, Terraform will correctly handle these attributes without needing the extra lifecycle configuration. Keep an eye on upcoming releases for this change.
### Step 4: Update Terraform State

Expand All @@ -179,6 +188,12 @@ After removing the `table` blocks and the table/topic specific attributes from y

After applying the changes, verify that your tables are still correctly set up in Materialize by checking the table definitions using Materialize's SQL commands.

For a more detailed view of a specific table, you can use the `SHOW CREATE TABLE` command:

```sql
SHOW CREATE TABLE materialize.public.mysql_table1_from_source;
```

## Importing Existing Tables

To import existing tables into your Terraform state, use the following command:
Expand Down
2 changes: 1 addition & 1 deletion docs/resources/table_grant.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,4 +53,4 @@ Import is supported using the following syntax:
terraform import materialize_table_grant.example <region>:GRANT|TABLE|<table_id>|<role_id>|<privilege>

# The region is the region where the database is located (e.g. aws/us-east-1)
```
```
2 changes: 1 addition & 1 deletion pkg/materialize/source_table_kafka.go
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,7 @@ func (b *SourceTableKafkaBuilder) Create() error {

if b.format.Protobuf != nil {
if b.format.Protobuf.SchemaRegistryConnection.Name != "" && b.format.Protobuf.MessageName != "" {
options = append(options, fmt.Sprintf(`FORMAT PROTOBUF MESSAGE '%s' USING CONFLUENT SCHEMA REGISTRY CONNECTION %s`, b.format.Protobuf.MessageName, QualifiedName(b.format.Protobuf.SchemaRegistryConnection.DatabaseName, b.format.Protobuf.SchemaRegistryConnection.SchemaName, b.format.Protobuf.SchemaRegistryConnection.Name)))
options = append(options, fmt.Sprintf(`FORMAT PROTOBUF MESSAGE %s USING CONFLUENT SCHEMA REGISTRY CONNECTION %s`, QuoteString(b.format.Protobuf.MessageName), QualifiedName(b.format.Protobuf.SchemaRegistryConnection.DatabaseName, b.format.Protobuf.SchemaRegistryConnection.SchemaName, b.format.Protobuf.SchemaRegistryConnection.Name)))
} else if b.format.Protobuf.SchemaRegistryConnection.Name != "" {
options = append(options, fmt.Sprintf(`FORMAT PROTOBUF USING CONFLUENT SCHEMA REGISTRY CONNECTION %s`, QualifiedName(b.format.Protobuf.SchemaRegistryConnection.DatabaseName, b.format.Protobuf.SchemaRegistryConnection.SchemaName, b.format.Protobuf.SchemaRegistryConnection.Name)))
}
Expand Down
17 changes: 16 additions & 1 deletion templates/guides/materialize_source_table.md.tmpl
Original file line number Diff line number Diff line change
@@ -1,3 +1,12 @@
---
{{ printf "# generated by https://github.com/hashicorp/terraform-plugin-docs" }}
{{ printf "# template file: templates/guides/materialize_source_table.md.tmpl" }}
page_title: "Source Table Migration Guide"
subcategory: ""
description: |-
Guide for migrating to the new materialize_source_table_{source_type} resources.
---

# Source versioning: migrating to `materialize_source_table_{source_type}` Resource

In previous versions of the Materialize Terraform provider, source tables were defined within the source resource itself and were considered subsources of the source rather than separate entities.
Expand Down Expand Up @@ -169,7 +178,7 @@ resource "materialize_source_kafka" "kafka_source" {

In the `lifecycle` block, add the `ignore_changes` meta-argument to prevent Terraform from trying to update these attributes during subsequent applies, that way Terraform won't try to update these values based on incomplete information from the state as they will no longer be defined in the source resource itself but in the new `materialize_source_table_{source_type}` resources.

> Note: We will make the changes to those attributes a no-op, so the `ignore_changes` block will not be necessary.
> Note: Once the migration process is fully implemented on the Materialize side and the attributes will have to be updated as no-op in future versions of the provider. That way the `ignore_changes` block will no longer be required. At that point, Terraform will correctly handle these attributes without needing the extra lifecycle configuration. Keep an eye on upcoming releases for this change.

### Step 4: Update Terraform State

Expand All @@ -179,6 +188,12 @@ After removing the `table` blocks and the table/topic specific attributes from y

After applying the changes, verify that your tables are still correctly set up in Materialize by checking the table definitions using Materialize's SQL commands.

For a more detailed view of a specific table, you can use the `SHOW CREATE TABLE` command:

```sql
SHOW CREATE TABLE materialize.public.mysql_table1_from_source;
```

## Importing Existing Tables

To import existing tables into your Terraform state, use the following command:
Expand Down
2 changes: 1 addition & 1 deletion templates/resources/table_grant.md.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ description: |-

Import is supported using the following syntax:

{{ codefile "shell" (printf "%s%s%s" "examples/resources/" .Name "/import.sh") }}
{{ codefile "shell" (printf "%s%s%s" "examples/resources/" .Name "/import.sh") }}

0 comments on commit 232787a

Please sign in to comment.