opentelekomcloud_enterprise_vpn_gateway_v5 and its relevant API deletes bound EIPs even they are passed by id

[canaykin]

Terraform provider version

registry.terraform.io/opentelekomcloud/opentelekomcloud v1.36.31

Affected Resource(s)

• opentelekomcloud_enterprise_vpn_gateway_v5

Terraform Configuration Files

resource "opentelekomcloud_vpc_eip_v1" "eips" {
  count = var.network_type == "public" ? 2 : 0
  bandwidth {
    name        = "${var.name}-vpn-${count.index == 0 ? "primary" : "secondary"}"
    size        = var.eip_bandwidth
    share_type  = "PER"
    charge_mode = "traffic"
  }
  publicip {
    type = "5_bgp"
  }
  tags = var.tags
}

resource "opentelekomcloud_enterprise_vpn_gateway_v5" "gateway" {
  name               = var.name
  availability_zones = var.availability_zones
  flavor             = var.flavor
  attachment_type    = var.attachment_type
  network_type       = var.network_type
  vpc_id             = var.vpc_id
  local_subnets      = var.local_subnets
  connect_subnet     = var.connect_subnet
  er_id              = var.er_id
  ha_mode            = var.ha_mode
  access_vpc_id      = var.access_vpc_id
  access_subnet_id   = var.access_subnet_id
  asn                = var.asn

  dynamic "eip1" {
    for_each = var.network_type == "public" ? [opentelekomcloud_vpc_eip_v1.eips[0].id] : toset([])
    content {
      id = eip1.value
    }
  }
  dynamic "eip2" {
    for_each = var.network_type == "public" ? [opentelekomcloud_vpc_eip_v1.eips[1].id] : toset([])
    content {
      id = eip2.value
    }
  }

  tags = var.tags
}

Steps to Reproduce

• Create an opentelekomcloud_enterprise_vpn_gateway_v5 resource with existing EIPs via eip1 and eip2 block id parameter and opentelekomcloud_vpc_eip_v1 resource.
• Destroy opentelekomcloud_enterprise_vpn_gateway_v5 while keeping the opentelekomcloud_vpc_eip_v1 resources

Expected Behavior

Similar to ELB and ECS EIP association behavior, the destruction of the EIP should only take place if the resource responsible for creation is destroyed (i.e. only if autocreation is used). When the EIP is managed by a different terraform resource and bound to the target via a UUID reference, the referenced resource should not be destroyed.

As it stands, when using Terraform to provision the gateway, there is no real difference between letting the opentelekomcloud_enterprise_vpn_gateway_v5 autocreate the EIPs and using the eip.id parameter beyond the capability to split configuration into different places.

Actual Behavior

The bound EIPs get deleted by the API when DELETE /v5/<project_id>/vpn-gateways/<vgw_id> is called by the provider, regardless of how they were created (which is due to the behavior of the API rather than provider).

This results in the following error in the Terraform which in and of itself is not a big issue but the behavior in general has more significant impact as described in the impact section.

│ Error: error unbinding eip: <eip_id> to port: error unbinding port from EIP: Resource not found: [PUT https://vpc.eu-de.otc.t-systems.com/v1/<project_id>/publicips/<eip_id>], error message: {"code":"VPC.0504","message":"{\"NeutronError\":{\"message\":\"Floating IP <eip_id> could not be found\",\"type\":\"FloatingIPNotFound\",\"detail\":\"\"}}"}

Impact

Luckily, the EIP deletion can be prevented by unbinding the EIP to the gateway as the API only takes action if the EIPs are bound at the deletion time. Unfortunately, since there is no EIP association resource for the VPN gateway, there is no way to solve the problem on the user side, as Terraform has no capability to unbind the EIP by destroying the association prior to destroying the gateway.

The biggest issue is that the inconsistent and unexpected behavior of the resource caught me by surprise when testing the new module and will likely catch other people by surprise (possibly in a non test scenario). The UI is nice enough to give a large warning in a confirmation box but there are no warnings or safeguards in case of Terraform calling the API.

Based on my experience with the numerous customers, the VPN service often is used either with old-school hosting providers or with on-premise systems. These providers still rely mostly on ticket based manual configuration and as such, a VPN configuration changes can require days to be implemented by them.

This fact greatly exacerbates the impact of releasing the EIPs, as the resulting downtime from an error can last for days until it can be fixed by the customers by updating the VPN configuration on the remote side.

Potential Remedy on the Provider Side

In my opinion, this is an API behavior problem where the enterprise VPN API does not follow the pattern of other, similar OTC resources. That being said, I believe it can still be remedied on the provider side by calling the EIP disassociate API before calling the DELETE /v5/<project_id>/vpn-gateways/<vgw_id>.

I have tested it by calling the EIP disassociate API (via Postman, not with Terraform) to see if this fixes the problem and the enterprise VPN was kind enough to not delete the EIP when it was disassociated from the gateway via this API.

The required parameters for the disassociate request are project_id and eip_id, which are both available to the provider via the provider config and the state of opentelekomcloud_enterprise_vpn_gateway_v5, so while I am not well versed on the provider development, it should in-theory be possible unbind it, if the EIP is passed by reference.

Thank you very much in advance for your effort and looking forward for your responses.
Can.