Skip to content

Convert DCL resources to handwritten#16759

Draft
SirGitsalot wants to merge 7 commits intoGoogleCloudPlatform:mainfrom
SirGitsalot:handwritten-dcl
Draft

Convert DCL resources to handwritten#16759
SirGitsalot wants to merge 7 commits intoGoogleCloudPlatform:mainfrom
SirGitsalot:handwritten-dcl

Conversation

@SirGitsalot
Copy link
Member

This is a very large change that should (assuming everything is correct!) have no effect on the downstreams. The affected resources are:

google_apikeys_key
google_assured_workloads_workload
google_cloudbuild_worker_pool
google_clouddeploy_delivery_pipeline
google_clouddeploy_target
google_container_aws_cluster
google_container_aws_node_pool
google_container_azure_client
google_container_azure_cluster
google_container_azure_node_pool
google_dataplex_asset
google_dataplex_lake
google_dataplex_zone
google_dataproc_workflow_template
google_firebaserules_release
google_firebaserules_ruleset
google_gke_hub_feature_membership
google_recaptcha_enterprise_key

The changes here can be broken down into a few groups:

  • The first group was by running tpgtools one last time but pointing the output back to magic-modules/mmv1/terraform/services/... instead of the downstreams. That's what produced the resource_[name].go[.tmpl], resource_[name]_generated_test.go[.tmpl], and resource_[name]_sweeper.go[.tmpl] files. Looking at the provider diffs for these should (and does!) show only changes from Type: DCL to Type: Handwritten and import path changes.

  • Similarly, docs were generated one last time, redirected into magic-modules/mmv1/terraform/website/docs/r/.

  • github.com/GoogleCloudPlatform/declarative-resource-client-library is no longer a dependency. Shared code from its dcl subdirectory was copied into magic-modules/mmv1/terraform/tpgdclresource. Product-specific and resource-specific code (client.go, [resource].go, and [resource]_internal.go) were moved into the appropriate directory under magic-modules/mmv1/terraform/services/. Reviewing the PR myself I see that [resource]_schema.go was also copied, but I'm pretty sure that's not needed so I'll try removing those. The DCL kept the GA and beta versions of these files in separate directories. In their Magic Modules incarnation, they're now single file templates that have been merged with {{- if ne $.TargetVersionName "ga" }} separating diffs. For files where GA and beta were the same, templates aren't used.

  • The root Makefile no longer touches the tpgtools directory.

  • Magic Modules developer docs have been updated to remove references to the DCL.

  • The tpgtools directory is not deleted in this PR because there's at least one lingering reference that I'm not sure what to replace with. Plus, this PR is already ginormous and it doesn't hurt to delete the directory in a subsequent PR.

I'm opening this PR in draft mode to make a self review easier, and I already see some changes that need to be made ( the aforementioned [resource]_schema.go removal, plus updating magic-modules/mmv1/terraform/go.[mod|sum] to actually remove the github.com/GoogleCloudPlatform/declarative-resource-client-library dep).


@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 196 files changed, 134288 insertions(+), 2357 deletions(-))
google-beta provider: Diff ( 196 files changed, 140146 insertions(+), 2618 deletions(-))

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 6120
Passed tests: 5458
Skipped tests: 639
Affected tests: 23

Click here to see the affected service packages

All service packages are affected

Action taken

Found 23 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccAlloydbCluster_maximumFields
  • TestAccApiGatewayGateway_apigatewayGatewayBasicExampleUpdated
  • TestAccApigeeEnvironmentKeyvaluemapsEntries_apigeeEnvironmentKeyvaluemapsEntriesTestExample
  • TestAccApigeeEnvironment_apigeeEnvironmentUpdateTest
  • TestAccApigeeOrganization_apigeeOrganizationCloudBasicDataResidencyTestExample
  • TestAccApigeeOrganization_apigeeOrganizationCloudFullTestExample
  • TestAccApphubServiceProjectAttachment_serviceProjectAttachmentFullExample
  • TestAccBigqueryConnectionConnectionIamPolicyGenerated
  • TestAccBigqueryDataTransferConfig__service_account
  • TestAccChronicleWatchlist_chronicleWatchlistBasicExample
  • TestAccContainerAwsNodePool_BasicHandWritten
  • TestAccContainerAwsNodePool_BetaBasicHandWritten
  • TestAccContainerAzureNodePool_BasicHandWritten
  • TestAccContainerAzureNodePool_BetaBasicHandWritten
  • TestAccDataSourceGoogleGKEHub2MembershipBinding_basic
  • TestAccDataformConfig_dataformConfigWithKmsKeyExample
  • TestAccDataformConfig_update
  • TestAccGKEHubFeatureMembership_gkehubFeatureAcmAllFields
  • TestAccGKEHubFeatureMembership_gkehubFeatureAcmOci
  • TestAccGKEHubFeatureMembership_gkehubFeatureAcmUpdate
  • TestAccGKEHubFeatureMembership_gkehubFeatureMesh
  • TestAccGKEHubFeatureMembership_gkehubFeaturePolicyController
  • TestAccSecurityCenterV2OrganizationBigQueryExportConfig_basic

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🟢 Tests passed during RECORDING mode:
TestAccAlloydbCluster_maximumFields [Debug log]
TestAccApiGatewayGateway_apigatewayGatewayBasicExampleUpdated [Debug log]
TestAccApigeeEnvironmentKeyvaluemapsEntries_apigeeEnvironmentKeyvaluemapsEntriesTestExample [Debug log]
TestAccApigeeEnvironment_apigeeEnvironmentUpdateTest [Debug log]
TestAccApigeeOrganization_apigeeOrganizationCloudBasicDataResidencyTestExample [Debug log]
TestAccApigeeOrganization_apigeeOrganizationCloudFullTestExample [Debug log]
TestAccApphubServiceProjectAttachment_serviceProjectAttachmentFullExample [Debug log]
TestAccBigqueryConnectionConnectionIamPolicyGenerated [Debug log]
TestAccBigqueryDataTransferConfig__booleanParam [Debug log]
TestAccBigqueryDataTransferConfig__service_account [Debug log]
TestAccBigqueryDataTransferConfig__update_params [Debug log]
TestAccBigqueryDataTransferConfig__update_service_account [Debug log]
TestAccChronicleWatchlist_chronicleWatchlistBasicExample [Debug log]
TestAccDataSourceGoogleGKEHub2MembershipBinding_basic [Debug log]
TestAccDataformConfig_dataformConfigWithKmsKeyExample [Debug log]
TestAccDataformConfig_update [Debug log]
TestAccSecurityCenterV2OrganizationBigQueryExportConfig_basic [Debug log]

🔴 Tests failed when rerunning REPLAYING mode:
TestAccDataformConfig_update [Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.


🔴 Tests failed during RECORDING mode:
TestAccContainerAwsNodePool_BasicHandWritten [Error message] [Debug log]
TestAccContainerAwsNodePool_BetaBasicHandWritten [Error message] [Debug log]
TestAccContainerAzureNodePool_BasicHandWritten [Error message] [Debug log]
TestAccContainerAzureNodePool_BetaBasicHandWritten [Error message] [Debug log]
TestAccGKEHubFeatureMembership_gkehubFeatureAcmAllFields [Error message] [Debug log]
TestAccGKEHubFeatureMembership_gkehubFeatureAcmOci [Error message] [Debug log]
TestAccGKEHubFeatureMembership_gkehubFeatureAcmUpdate [Error message] [Debug log]
TestAccGKEHubFeatureMembership_gkehubFeatureMesh [Error message] [Debug log]
TestAccGKEHubFeatureMembership_gkehubFeaturePolicyController [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants