Skip to content

Commit 5acd9e7

Browse files
omerdemirokcarabasdaniel
authored andcommitted
Create backlinks and forward links in GCP dynamic adapters (#2807)
# Add Bidirectional Hierarchical Linking for GCP Resources ## Overview This PR implements comprehensive support for bidirectional linking between hierarchical GCP resources (parent-child relationships). It enables automatic discovery of relationships in both directions: - **Backlinks:** Child resources can link to their parent (e.g., Database → Instance) - **Forward links:** Parent resources can discover all their children (e.g., Instance → All Databases) ## Key Changes ### 1. Framework Enhancement **File:** `sources/gcp/shared/linker.go` Added `IsParentToChild` flag support to enable parent-to-child forward linking: - Detects the `IsParentToChild` flag on blast propagation impacts - Automatically removes the last unique attribute key (child identifier) when creating forward links - Changes query method from GET to SEARCH to discover all child resources - Fixes variable shadowing for regional/zonal scope handling **Example:** A Spanner Instance can now link to all its databases using SEARCH with just the instance name, without needing to know specific database names. ### 2. Backlinks Implemented (Child → Parent via GET) Uses the `name` field to extract parent identifiers from hierarchical resource names: - **BigTableAdminCluster** → BigTableAdminInstance - **ContainerNodePool** → ContainerCluster - **SpannerBackup** → SpannerInstance - **SpannerDatabase** → SpannerInstance _(initial implementation)_ **How it works:** The framework extracts the parent identifier from the child's name (e.g., `projects/p/instances/i/databases/d` → extracts `i`) and creates a GET query to the parent resource. ### 3. Forward Links Implemented (Parent → Child via SEARCH) Uses the `IsParentToChild: true` flag to create SEARCH queries that discover all children: - **BigTableAdminInstance** → BigTableAdminCluster - **ArtifactRegistryRepository** → ArtifactRegistryDockerImage - **ContainerCluster** → ContainerNodePool - **RunService** → RunRevision - **SpannerInstance** → SpannerDatabase - **SQLAdminInstance** → SQLAdminBackupRun **How it works:** The framework uses the parent's name to create a SEARCH query that returns all child resources belonging to that parent. ### 4. Documentation **File:** `sources/gcp/dynamic/adapters/.cursor/rules/dynamic-adapter-creation.mdc` Added comprehensive documentation (143 lines) covering: #### Backlink Pattern - Concept explanation and use cases - Complete code example using Spanner Database → Instance - Step-by-step framework behavior breakdown - Test patterns for verification - Guidelines on when to use this pattern #### Forward Link Pattern - Concept explanation and use cases - Complete code example using Spanner Instance → Databases - Framework behavior with `IsParentToChild` flag - Critical requirements (child must support SEARCH) - Test patterns for verification - Comparison table: Backlink vs Forward Link ### 5. Tests Updated - `big-table-admin-cluster_test.go`: Added backlink test case - `spanner-instance_test.go`: Added forward link test case ## Validation ✅ All tests passing: `go test -race ./sources/gcp/... -v -timeout 5m` ✅ Linting clean: `golangci-lint run ./sources/gcp/... --timeout 5m` ## Impact This PR enables complete bidirectional relationship discovery for hierarchical GCP resources, improving: - **Dependency tracking:** Understanding which children depend on a parent - **Impact analysis:** Discovering all resources affected by parent changes - **Infrastructure visualization:** Building complete resource graphs with parent-child relationships --- Would you like me to make any adjustments to this PR description? --------- Co-authored-by: carabasdaniel <[email protected]> GitOrigin-RevId: c8d839894d83a800f8c5d9f75019e59847bdc920
1 parent 3248b7c commit 5acd9e7

14 files changed

+384
-81
lines changed

sources/gcp/dynamic/adapter_test.go

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -842,6 +842,17 @@ func TestAdapter(t *testing.T) {
842842
Out: false,
843843
},
844844
},
845+
{
846+
// name field creates a backlink to the Spanner instance
847+
ExpectedType: gcpshared.SpannerInstance.String(),
848+
ExpectedMethod: sdp.QueryMethod_GET,
849+
ExpectedQuery: instanceName,
850+
ExpectedScope: projectID,
851+
ExpectedBlastPropagation: &sdp.BlastPropagation{
852+
In: true,
853+
Out: false,
854+
},
855+
},
845856
}
846857

847858
shared.RunStaticTests(t, adapter, sdpItem, queryTests)

sources/gcp/dynamic/adapters/.cursor/rules/dynamic-adapter-creation.mdc

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,6 +179,151 @@ blastPropagation: map[string]*gcpshared.Impact{
179179
- **If no adapter exists**: Create the SDP item type definition in `sources/gcp/shared/item-types.go` and `models.go` as if the adapter exists, then link to it
180180
- **Follow naming**: `gcp-<api>-<resource>` for new adapter types
181181

182+
#### Creating Backlinks from Child to Parent Resources
183+
184+
When a resource name contains hierarchical information (e.g., a database name includes its parent instance), you can create a backlink using the `name` field. The framework will automatically extract the parent resource identifier and create the appropriate linked item query.
185+
186+
**Use Case**: Child resources that reference their parent in their name structure.
187+
188+
**Example: Spanner Database to Instance Backlink**
189+
190+
A Spanner database name has the format: `projects/{project}/instances/{instance}/databases/{database}`
191+
192+
To create a backlink from the database to its parent instance:
193+
194+
```go
195+
// In sources/gcp/shared/blast-propagations.go
196+
SpannerDatabase: {
197+
// ... other blast propagations ...
198+
199+
// This is a backlink to instance.
200+
// Framework will extract the instance name and create the linked item query with GET
201+
"name": {
202+
Description: "If the Spanner Instance is deleted or updated: The Database may become invalid or inaccessible. If the Database is updated: The instance remains unaffected.",
203+
ToSDPItemType: SpannerInstance,
204+
BlastPropagation: impactInOnly,
205+
},
206+
}
207+
```
208+
209+
**How It Works:**
210+
1. The framework reads the `name` field value (e.g., `projects/my-project/instances/test-instance/databases/my-db`)
211+
2. It extracts the parent instance identifier (`test-instance`)
212+
3. It creates a GET query for the parent SpannerInstance
213+
4. The linked item query will have:
214+
- Type: `gcp-spanner-instance`
215+
- Method: GET
216+
- Query: `test-instance` (extracted from database name)
217+
- Scope: Same project as the database
218+
219+
**Testing the Backlink:**
220+
221+
When adding backlink blast propagations, verify they work correctly in tests:
222+
223+
```go
224+
// In adapter_test.go
225+
{
226+
// name field creates a backlink to the Spanner instance
227+
ExpectedType: gcpshared.SpannerInstance.String(),
228+
ExpectedMethod: sdp.QueryMethod_GET,
229+
ExpectedQuery: instanceName,
230+
ExpectedScope: projectID,
231+
ExpectedBlastPropagation: &sdp.BlastPropagation{
232+
In: true,
233+
Out: false,
234+
},
235+
}
236+
```
237+
238+
**When to Use This Pattern:**
239+
- Child resources with hierarchical naming (database → instance, subnet → network)
240+
- Parent resource is part of the child's name/path
241+
- Relationship is one-way (child depends on parent, but parent doesn't depend on specific child)
242+
- Use `impactInOnly` blast propagation (parent changes affect child, but not vice versa)
243+
244+
#### Creating Forward Links from Parent to Child Resources
245+
246+
When a parent resource needs to link to all its child resources, you can use the `IsParentToChild` flag to create a forward link using SEARCH. This is the inverse pattern of the backlink described above.
247+
248+
**Use Case**: Parent resources that need to discover and link to all their child resources (e.g., instance → all databases).
249+
250+
**Example: Spanner Instance to Databases Forward Link**
251+
252+
A Spanner instance needs to link to all databases that belong to it. Since the instance doesn't have the database names, we use SEARCH to find all databases for that instance.
253+
254+
To create a forward link from the instance to its databases:
255+
256+
```go
257+
// In the adapter file (e.g., spanner-instance.go)
258+
var spannerInstanceAdapter = registerableAdapter{
259+
// ... other fields ...
260+
261+
blastPropagation: map[string]*gcpshared.Impact{
262+
// This a link from parent to child via SEARCH
263+
// We need to make sure that the linked item supports `SEARCH` method for the `instance` name.
264+
"name": {
265+
ToSDPItemType: gcpshared.SpannerDatabase,
266+
Description: "If the Spanner Instance is deleted or updated: All associated databases may become invalid or inaccessible. If a database is updated: The instance remains unaffected.",
267+
BlastPropagation: &sdp.BlastPropagation{
268+
In: false,
269+
Out: true,
270+
},
271+
IsParentToChild: true,
272+
},
273+
},
274+
}
275+
```
276+
277+
**How It Works:**
278+
1. The framework detects `IsParentToChild: true` on the blast propagation
279+
2. It modifies the unique attribute keys by removing the last element (the child identifier)
280+
- For databases: `["instances", "databases"]` becomes `["instances"]`
281+
3. It extracts only the parent identifier from the resource name (e.g., `test-instance`)
282+
4. It creates a SEARCH query (not GET) to find all child resources
283+
5. The linked item query will have:
284+
- Type: `gcp-spanner-database`
285+
- Method: SEARCH
286+
- Query: `test-instance` (the instance name)
287+
- Scope: Same project as the instance
288+
289+
**Testing the Forward Link:**
290+
291+
When adding parent-to-child blast propagations, verify they work correctly in tests:
292+
293+
```go
294+
// In adapter_test.go (e.g., spanner-instance_test.go)
295+
{
296+
ExpectedType: gcpshared.SpannerDatabase.String(),
297+
ExpectedMethod: sdp.QueryMethod_SEARCH,
298+
ExpectedQuery: instanceName,
299+
ExpectedScope: projectID,
300+
ExpectedBlastPropagation: &sdp.BlastPropagation{
301+
In: false,
302+
Out: true,
303+
},
304+
}
305+
```
306+
307+
**Critical Requirements:**
308+
- **Child adapter must support SEARCH**: The child resource type must implement the SEARCH method
309+
- **Query parameter must match**: The child's SEARCH method must accept the parent identifier as a search parameter
310+
- **Set `IsParentToChild: true`**: This flag triggers the special SEARCH behavior
311+
312+
**When to Use This Pattern:**
313+
- Parent resources that need to discover all their children (instance → databases, network → subnets)
314+
- The parent's name/identifier is sufficient to search for children
315+
- The child adapter supports SEARCH method
316+
317+
**Comparison: Backlink vs Forward Link**
318+
319+
| Aspect | Backlink (Child → Parent) | Forward Link (Parent → Child) |
320+
|--------|---------------------------|------------------------------|
321+
| Direction | Child points to parent | Parent discovers children |
322+
| Method | GET | SEARCH |
323+
| Flag | No special flag | `IsParentToChild: true` |
324+
| Use Case | Database → Instance | Instance → All Databases |
325+
| Query | Single parent identifier | Parent identifier searches all children |
326+
182327
### 6. Special Considerations
183328

184329
#### NameSelector

sources/gcp/dynamic/adapters/artifact-registry-repository.go

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,17 @@ var artifactRegistryRepositoryAdapter = registerableAdapter{ //nolint:unused
2323
},
2424
blastPropagation: map[string]*gcpshared.Impact{
2525
"kmsKeyName": gcpshared.CryptoKeyImpactInOnly,
26+
// Forward link from parent to child via SEARCH
27+
// Link to all docker images in this repository
28+
"name": {
29+
ToSDPItemType: gcpshared.ArtifactRegistryDockerImage,
30+
Description: "If the Artifact Registry Repository is deleted or updated: All associated Docker Images may become invalid or inaccessible. If a Docker Image is updated: The repository remains unaffected.",
31+
BlastPropagation: &sdp.BlastPropagation{
32+
In: false,
33+
Out: true,
34+
},
35+
IsParentToChild: true,
36+
},
2637
},
2738
terraformMapping: gcpshared.TerraformMapping{
2839
Reference: "https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/artifact_registry_repository#attributes-reference",

sources/gcp/dynamic/adapters/big-table-admin-cluster.go

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,18 @@ var bigTableAdminClusterAdapter = registerableAdapter{ //nolint:unused
2424
blastPropagation: map[string]*gcpshared.Impact{
2525
// Customer-managed encryption key protecting data in this cluster.
2626
"encryptionConfig.kmsKeyName": gcpshared.CryptoKeyImpactInOnly,
27+
// This is a backlink to instance.
28+
// Framework will extract the instance name and create the linked item query with GET
29+
// NOTE: We prioritize the backlink over a forward link to BigTableAdminBackup
30+
// because the backlink is more critical for understanding the cluster's dependencies.
31+
"name": {
32+
ToSDPItemType: gcpshared.BigTableAdminInstance,
33+
Description: "If the BigTableAdmin Instance is deleted or updated: The Cluster may become invalid or inaccessible. If the Cluster is updated: The instance remains unaffected.",
34+
BlastPropagation: &sdp.BlastPropagation{
35+
In: true,
36+
Out: false,
37+
},
38+
},
2739
},
2840
// No Terraform mapping
2941
}.Register()

sources/gcp/dynamic/adapters/big-table-admin-cluster_test.go

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,17 @@ func TestBigTableAdminCluster(t *testing.T) {
176176
Out: false,
177177
},
178178
},
179+
{
180+
// name field creates a backlink to the BigTable instance
181+
ExpectedType: gcpshared.BigTableAdminInstance.String(),
182+
ExpectedMethod: sdp.QueryMethod_GET,
183+
ExpectedQuery: instanceName,
184+
ExpectedScope: projectID,
185+
ExpectedBlastPropagation: &sdp.BlastPropagation{
186+
In: true,
187+
Out: false,
188+
},
189+
},
179190
}
180191

181192
shared.RunStaticTests(t, adapter, sdpItem, queryTests)

sources/gcp/dynamic/adapters/big-table-admin-instance.go

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,19 @@ var bigTableAdminInstanceAdapter = registerableAdapter{ //nolint:unused
2121
// TODO: https://linear.app/overmind/issue/ENG-631/investigate-how-we-can-add-health-status-for-supporting-items
2222
// state: https://cloud.google.com/bigtable/docs/reference/admin/rest/v2/projects.instances#State
2323
},
24-
// The Bigtable Instance does not contain any fields that would cause blast propagation.
25-
blastPropagation: map[string]*gcpshared.Impact{},
24+
blastPropagation: map[string]*gcpshared.Impact{
25+
// Forward link from parent to child via SEARCH
26+
// Link to all clusters in this instance (most fundamental infrastructure component)
27+
"name": {
28+
ToSDPItemType: gcpshared.BigTableAdminCluster,
29+
Description: "If the BigTableAdmin Instance is deleted or updated: All associated Clusters may become invalid or inaccessible. If a Cluster is updated: The instance remains unaffected.",
30+
BlastPropagation: &sdp.BlastPropagation{
31+
In: false,
32+
Out: true,
33+
},
34+
IsParentToChild: true,
35+
},
36+
},
2637
terraformMapping: gcpshared.TerraformMapping{
2738
Reference: "https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/bigtable_instance",
2839
Mappings: []*sdp.TerraformMapping{

sources/gcp/dynamic/adapters/container-cluster.go

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,17 @@ var _ = registerableAdapter{
5151
"userManagedKeysConfig.serviceAccountVerificationKeys": gcpshared.CryptoKeyVersionImpactInOnly,
5252
"userManagedKeysConfig.controlPlaneDiskEncryptionKey": gcpshared.CryptoKeyImpactInOnly,
5353
"userManagedKeysConfig.gkeopsEtcdBackupEncryptionKey": gcpshared.CryptoKeyImpactInOnly,
54+
// Forward link from parent to child via SEARCH
55+
// Link to all node pools in this cluster
56+
"name": {
57+
ToSDPItemType: gcpshared.ContainerNodePool,
58+
Description: "If the Container Cluster is deleted or updated: All associated Node Pools may become invalid or inaccessible. If a Node Pool is updated: The cluster remains unaffected.",
59+
BlastPropagation: &sdp.BlastPropagation{
60+
In: false,
61+
Out: true,
62+
},
63+
IsParentToChild: true,
64+
},
5465
},
5566
terraformMapping: gcpshared.TerraformMapping{
5667
Reference: "https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/container_cluster",

sources/gcp/dynamic/adapters/container-node-pool.go

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,13 @@ var containerNodePoolAdapter = registerableAdapter{ //nolint:unused
3636
// https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1/projects.locations.clusters.nodePools#NodePool.Status
3737
},
3838
blastPropagation: map[string]*gcpshared.Impact{
39+
// This is a backlink to cluster.
40+
// Framework will extract the cluster name and create the linked item query with GET
41+
"name": {
42+
ToSDPItemType: gcpshared.ContainerCluster,
43+
Description: "If the Container Cluster is deleted or updated: The Node Pool may become invalid or inaccessible. If the Node Pool is updated: The cluster remains unaffected.",
44+
BlastPropagation: gcpshared.ImpactInOnly,
45+
},
3946
"config.bootDiskKmsKey": gcpshared.CryptoKeyImpactInOnly,
4047
"config.serviceAccount": gcpshared.IAMServiceAccountImpactInOnly,
4148
"config.nodeGroup": {

sources/gcp/dynamic/adapters/run-service.go

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,17 @@ var _ = registerableAdapter{
8787
Description: "If the Cloud Run Service is deleted or updated: Traffic allocation to revisions will be lost. If revisions are updated: The service traffic configuration may need updates.",
8888
BlastPropagation: &sdp.BlastPropagation{Out: true},
8989
},
90+
// Forward link from parent to child via SEARCH
91+
// Link to all revisions in this service
92+
"name": {
93+
ToSDPItemType: gcpshared.RunRevision,
94+
Description: "If the Cloud Run Service is deleted or updated: All associated Revisions may become invalid or inaccessible. If a Revision is updated: The service remains unaffected.",
95+
BlastPropagation: &sdp.BlastPropagation{
96+
In:false,
97+
Out: true,
98+
},
99+
IsParentToChild: true,
100+
},
90101
},
91102
terraformMapping: gcpshared.TerraformMapping{
92103
Reference: "https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloud_run_v2_service",

sources/gcp/dynamic/adapters/spanner-instance.go

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,17 @@ var spannerInstanceAdapter = registerableAdapter{ //nolint:unused
3131
Out: false,
3232
},
3333
},
34+
// This is a link from parent to child via SEARCH
35+
// We need to make sure that the linked item supports `SEARCH` method for the `instance` name.
36+
"name": {
37+
ToSDPItemType: gcpshared.SpannerDatabase,
38+
Description: "If the Spanner Instance is deleted or updated: All associated databases may become invalid or inaccessible. If a database is updated: The instance remains unaffected.",
39+
BlastPropagation: &sdp.BlastPropagation{
40+
In: false,
41+
Out: true,
42+
},
43+
IsParentToChild: true,
44+
},
3445
},
3546
terraformMapping: gcpshared.TerraformMapping{
3647
Reference: "https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/spanner_instance",

0 commit comments

Comments
 (0)