Merge pull request #1 from fortinet/v1.0.0

v1.0.0 upload

.gitignore

@@ -0,0 +1,3 @@
+.terraform/*
+terraform.tfstate
+terraform.tfstate.backup

LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2023 Fortinet
+Copyright (c) 2022 Fortinet Public Cloud Team EMEA
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -1 +1,62 @@
-# terraform-google-fgt-ha-ap-lb
+# FortiGate Terraform module:
+## HA Active-Passive cluster (FGCP in load balancer sandwich)
+
+![architecture diagram](./docs/diagram.png)
+
+This terraform module can be used to deploy the base part of FortiGate reference architecture consisting of:
+- 2 FortiGate VM instances - preconfigured in FGCP Active-Passive cluster
+- zonal instance groups to be used later as components of backend services
+- internal load balancer resources in trusted (internal) network
+- backend service in external network (load balancer without frontends)
+- (optionally) external IP addresses and ELB frontends (forwarding rules)
+- cloud firewall rules opening ALL communication on untrusted and trusted networks
+- cloud firewall rules allowing cluster sync and administrative access
+- static external IP addresses for management bound to nic3 (port4) of FortiGates
+- Cloud NAT to allow traffic initiated by FGTs out
+- (optionally) Secret Manager secret with FGT API token
+
+### How to use this module
+We assume you have a working root module with proper Google provider configuration. If you don't - start by reading [Google Provider Configuration Reference](https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference).
+
+1. Create before you start (or define in your root terraform module) 4 VPC networks with one subnet in each. All subnets must be in the region where you want to deploy FortiGates and their CIDRs cannot overlap
+1. Copy license files (*.lic) to the root module folder if you plan to deploy BYOL version. If using BYOL version you also have to change the `image_family` or `image_name` variable (see [examples/licensing-byol](./examples/licensing-byol) for details)
+1. Reference this module in your code (eg. main.tf) to use it, eg.:
+    ```
+    module "fgt-ha" {  
+      source = "git::github.com/fortinet/terraform-google-fgt-ha-ap-lb"  
+    }
+    ```
+1. In the above module block provide the variables described in `variables.tf`. Only 2 variables are obligatory:
+    - `region` - name of the region to deploy to (zones will be selected automatically); it also indicates subnets to use
+    - `subnets` - list of 4 names of subnets already existing in the region to be used as external, internal, heartbeat and management networks.
+
+    but you might want to provide values also to some others:
+    - `zones` - list of 2 zones for FortiGate VMs. Always match these to your production workloads to avoid [inter-zone traffic fees](https://cloud.google.com/vpc/network-pricing). You can skip for proof-of-concept deployments and let the module automatically detect zones in the region.
+    - `license_files` - list of paths to 2 license (.lic) files to be applied to the FortiGates. If skipped, VMs will be deployed without license and you will have to apply them manually upon first connection. It is highly recommended to apply BYOL licenses during deployment.
+    - `prefix` - prefix to be added to the names of all created resources (defaults to "**fgt**")
+    - `labels` - map of [Google Cloud labels](https://cloud.google.com/compute/docs/labeling-resources) to be applied to VMs, disks and forwarding rules
+    - `admin_acl` - list of CIDRs allowed to access FortiGates' management interfaces (defaults to [0.0.0.0/0])
+    - `machine-type` - type of VM to use for deployment. Defaults to **e2-standard-4** which is a good (cheaper) choice for evaluation, but offers lower performance than n2 or c2 families.
+    - `image_family` or `image_name` - for selecting different firmware version or different licensing model. Defaults to newest 7.2 image with PAYG licensing (fortigate-72-payg)
+    - `frontends` - list of names to be used to create ELB frontends and EIPs. By default no frontends are created. Resource names will be prepended with the `var.prefix` and resource type.
+1. Run the deployment using the tool of your choice (eg. `terraform init; terraform apply` from command line)
+
+Examples can be found in [examples](examples) directory.
+
+### Licensing
+FortiGates in GCP can be licensed in 3 ways:
+1. [PAYG](examples/licensing-payg) - paid per each hour of use via Google Cloud Marketplace after you deploy. This is the default setting for this module and you don't need to change anything to use it.
+2. [BYOL](examples/licensing-byol) - pay upfront via Fortinet Reseller. You will receive the license activation code, which needs to be registered in [Fortinet Support Portal](https://support.fortinet.com). After activation you will receive **.lic** license files which you need to add to your terraform deployment code and reference using `license_files` input variable. You will also need to change the `image_family` or `image_name` variable to a byol image.
+3. [FortiFlex (FlexVM)](examples/licensing-flex) - if you have an Enterprise Agreement with Fortinet and use FortiFlex portal, you will have to change the deployed image to BYOL. License tokens can be passed to the module using `flexvm_tokens` variable. Note that tokens cannot be re-used. If you need to re-deploy cluster with the same licenses you need to regenerate tokens in FlexVM portal.
+
+### Connecting to management interface
+After deployment you can access management interfaces of both instances directly through their public management addresses listed in `fgt_mgmt_eips` module output. By default you can access management interfaces from any network, but the access can (and should!) be restricted by using the `admin_acl` module variable. The initial password is set to the instance id of the primary fortigate (listed in module output `fgt_password`) and you will have to change it upon first login.
+
+### Configuration
+* [External IP addresses](examples/public-addresses-elb-frontend)
+* [Using ARM-based machine type (T2A family)](examples/arm-based-machine-type)
+* [Selecting proper boot image](docs/images.md)
+* [GVNIC driver and custom images](examples/gvnic-custom-image)
+
+### Customizations
+1. all addresses are static but picked automatically from the pool of available addresses for a given subnet. modify addresses.tf to manually indicate addresses you want to assign.

addresses.tf

@@ -0,0 +1,58 @@
+# All addresses are static but automatically assigned from the subnets.
+# This module does not support manual selection of addresses. Modify this file if you want to decide
+# which address will be assigned to what
+
+resource "google_compute_address" "mgmt_pub" {
+  count                  = 2
+
+  region                 = var.region
+  name                   = "${local.prefix}eip${count.index+1}-mgmt-${local.region_short}"
+}
+
+resource "google_compute_address" "ext_priv" {
+  count                  = 2
+
+  name                   = "${local.prefix}ip${count.index+1}-ext-${local.region_short}"
+  region                 = var.region
+  address_type           = "INTERNAL"
+  subnetwork             = data.google_compute_subnetwork.subnets[0].id
+}
+
+resource "google_compute_address" "int_priv" {
+  count                  = 2
+
+  name                   = "${local.prefix}ip${count.index+1}-int-${local.region_short}"
+  region                 = var.region
+  address_type           = "INTERNAL"
+  subnetwork             = data.google_compute_subnetwork.subnets[1].id
+}
+
+resource "google_compute_address" "ilb" {
+  name                   = "${local.prefix}ip-ilb-${local.region_short}"
+  region                 = var.region
+  address_type           = "INTERNAL"
+  subnetwork             = data.google_compute_subnetwork.subnets[1].id
+
+  # move ILB addresses after FGT addresses for more consistent address assignment
+  depends_on = [
+    google_compute_address.int_priv
+  ]
+}
+
+resource "google_compute_address" "hasync_priv" {
+  count                  = 2
+
+  name                   = "${local.prefix}ip${count.index+1}-hasync-${local.region_short}"
+  region                 = var.region
+  address_type           = "INTERNAL"
+  subnetwork             = data.google_compute_subnetwork.subnets[2].id
+}
+
+resource "google_compute_address" "mgmt_priv" {
+  count                  = 2
+
+  name                   = "${local.prefix}ip${count.index+1}-mgmt-${local.region_short}"
+  region                 = var.region
+  address_type           = "INTERNAL"
+  subnetwork             = data.google_compute_subnetwork.subnets[3].id
+}

cloudnat.tf

@@ -0,0 +1,24 @@
+# Cloud NAT is used for outbound connectivity from FortiGate instances.
+# it will be used for all connections initiated by FortiGates (eg. license
+# entitlement checks, signature updates, etc.), as well as for forwarded
+# connections if SNAT is set to port1 interface IP (default).
+
+# Cloud NAT will use ephemeral external IP
+
+resource "google_compute_router" "nat_router" {
+  name                   = "${local.prefix}cr-cloudnat-${local.region_short}"
+  region                 = var.region
+  network                = data.google_compute_subnetwork.subnets[0].network
+}
+
+resource "google_compute_router_nat" "cloud_nat" {
+  name                   = "${local.prefix}nat-cloudnat-${local.region_short}"
+  router                 = google_compute_router.nat_router.name
+  region                 = var.region
+  nat_ip_allocate_option = "AUTO_ONLY"
+  source_subnetwork_ip_ranges_to_nat = "LIST_OF_SUBNETWORKS"
+  subnetwork {
+    name                    = data.google_compute_subnetwork.subnets[0].self_link
+    source_ip_ranges_to_nat = ["ALL_IP_RANGES"]
+  }
+}

docs/images.md

@@ -0,0 +1,79 @@
+# Selecting base image for FortiGate instances
+
+By default, the FortiGates will be deployed with the newest 7.2 version and PAYG licensing on Intel-based platform. If you want to use a different firmware version, licensing or ARM-based VM instance - you must change the image by declaring one or more of the module variables: `image_name`, `image_family`, `image_project`.
+
+## Using family_name
+*Family name* is the easiest way to select the latest images published by Fortinet and supports both BYOL and PAYG deployments. Image *families* are a feature of Google Compute automatically selecting the latest image of a given family. As Fortinet customers often decide to not use the latest firmware branch, images are published with separate families for each main branch (6.4, 7.0, 7.2). You will also need to decide if the image license should be paid via Google Cloud Marketplace per time used (PAYG) or upfront (BYOL or FlexVM license). ARM-based images need to be declared using "**fortigate-arm64**" prefix instead of "**fortigate**".
+
+At the time of writing the following family names are available:
+- fortinet-64-byol
+- fortinet-64-payg
+- fortinet-70-byol
+- fortinet-70-payg
+- fortinet-72-byol
+- fortinet-72-payg
+- fortinet-arm64-72-byol
+- fortinet-arm64-72-payg
+
+To indicate family name pass it to the module as `family_name` variable. Eg.:
+
+```
+module fgt_ha {
+  source        = "git::github.com/fortinet/terraform-google-fgt-ha-ap-lb"
+
+  region        = "us-central1"
+  subnets       = [ "external", "internal", "hasync", "mgmt" ]
+  image_family  = "fortinet-70-byol"
+}
+```
+
+> **Note**: family_name gets mapped to the newest image every time terraform template is run. Re-running the plan after deployment will detect a drift if a new firmware version was published in the meantime.
+
+## Using image_name
+Indicating explicit image name is useful for selecting precisely the firmware version and locking it down in case your terraform code is re-run often and drift problem described in previous section might be an issue.
+
+To find the image name list images from Fortinet's public fortigcp-project-001 project:
+
+```
+gcloud compute images list --project fortigcp-project-001 --no-standard-images | grep fortinet-fgt
+```
+
+and provide desired image name as `image_name` module variable. Eg.:
+
+```
+module fgt_ha {
+  source        = "git::github.com/fortinet/terraform-google-fgt-ha-ap-lb"
+
+  region        = "us-central1"
+  subnets       = [ "external", "internal", "hasync", "mgmt" ]
+  image_name    = "fortinet-fgtondemand-6410-20220829-001-w-license"
+}
+```
+
+## Custom images
+If your deployment is using custom images, either derived from public ones or running an interim firmware build, pass to the module both `image_name` and `image_project` variables. Eg.:
+
+```
+module fgt_ha {
+  source        = "git::github.com/fortinet/terraform-google-fgt-ha-ap-lb"
+
+  region        = "us-central1"
+  subnets       = [ "external", "internal", "hasync", "mgmt" ]
+  image_name    = "my-fgt-image"
+  image_project = "my-project"
+}
+```
+
+***NOTE:*** if using a custom image with GVNIC support, you can use GVNIC driver by setting `nic_type`:
+
+```
+module fgt_ha {
+  source        = "git::github.com/fortinet/terraform-google-fgt-ha-ap-lb"
+
+  region        = "us-central1"
+  subnets       = [ "external", "internal", "hasync", "mgmt" ]
+  image_name    = "my-fgt-image"
+  image_project = "my-project"
+  nic_type      = "GVNIC"
+}
+```

elb.tf

@@ -0,0 +1,81 @@
+# External Load Balancer is used to drive traffic from Internet to FGT cluster
+# but can be also used for outbound traffic
+#
+# var.frontends can contain both names for new addresses and values for existing addresses
+# manipulations in locals below make the module distinguish between the two
+
+
+locals {
+  # split input frontends list into existing and to-be-created EIPs
+  in_eip_new = [ for addr in var.frontends : addr if !can(regex( "^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}$", addr ))]
+  in_eip_existing = [ for addr in var.frontends : addr if can(regex( "^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}$", addr ))]
+
+  # format existing EIP list into mapping by name, skip non-existing addresses, skip IN_USE addresses
+  eip_existing_existing = { for addr,info in data.google_compute_addresses.existing : addr => info if length(info.addresses)>0 }
+  eip_existing = { for addr,info in local.eip_existing_existing : trimprefix(info.addresses[0].name, local.prefix) => addr if info.addresses[0].status!="IN_USE"}
+
+  # format new EIP list into mapping by name
+  eip_new = {for name,info in google_compute_address.new_eip : name => info.address }
+
+  eip_all = merge( local.eip_new, local.eip_existing )
+}
+
+# pull data about existing EIPs to be assigned to the cluster for:
+# - sanity check if EIP is available to use
+# - getting EIP name for resource naming
+data "google_compute_addresses" "existing" {
+  for_each = toset(local.in_eip_existing)
+
+  region = var.region
+  filter = "address=\"${each.value}\""
+
+# NOTE: in contrary to documentation lifecycle is not supported for data.
+#       unavailable addresses will be silently ignored
+#  lifecycle {
+#    postcondition {
+#      condition = length( self.addresses )>0
+#      error_message = "Address ${each.value} was not found in region ${var.region}."
+#    }
+#  }
+}
+
+resource "google_compute_address" "new_eip" {
+  for_each = toset(local.in_eip_new)
+
+  name                  = "${local.prefix}eip-${each.value}"
+  region                = var.region
+  address_type          = "EXTERNAL"
+}
+
+resource "google_compute_forwarding_rule" "frontends" {
+  for_each              = local.eip_all
+
+  name                  = "${local.prefix}fr-${each.key}"
+  region                = var.region
+  ip_address            = each.value
+  ip_protocol           = "L3_DEFAULT"
+  all_ports             = true
+  load_balancing_scheme = "EXTERNAL"
+  backend_service       = google_compute_region_backend_service.elb_bes.self_link
+  labels                = var.labels
+}
+
+resource "google_compute_region_backend_service" "elb_bes" {
+  provider               = google-beta
+  name                   = "${local.prefix}bes-elb-${local.region_short}"
+  region                 = var.region
+  load_balancing_scheme  = "EXTERNAL"
+  protocol               = "UNSPECIFIED"
+
+  backend {
+    group                = google_compute_instance_group.fgt-umigs[0].self_link
+  }
+  backend {
+    group                = google_compute_instance_group.fgt-umigs[1].self_link
+  }
+
+  health_checks          = [google_compute_region_health_check.health_check.self_link]
+  connection_tracking_policy {
+    connection_persistence_on_unhealthy_backends = "NEVER_PERSIST"
+  }
+}

examples/api-token/README.md

@@ -0,0 +1,5 @@
+# Example: API token
+
+This example will create on FortiGates an API user account and will assign it a 'prof_admin' role and a random token generated during terraform deployment. The token will be stored in GCP Secret Manager and in module outputs. Access will be restricted to 10.0.0.0/24 subnet.
+
+Note: token will be visible to anyone having access to terraform state file, it is also included in VM instance metadata.

examples/api-token/main.tf

@@ -0,0 +1,10 @@
+module "fgt_ha" {
+  source        = "git::github.com/40net-cloud/fortigate-gcp-ha-ap-lb-terraform?ref=v1.0.0"
+
+  region        = "us-central1"
+  subnets       = [ "external", "internal", "hasync", "mgmt" ]
+
+  api_accprofile        = "prof_admin"
+  api_acl               = ["10.0.0.0/24"]
+  api_token_secret_name = "fgt-api-secret"
+}

examples/arm-based-machine-type/README.md

@@ -0,0 +1,28 @@
+# Module feature: ARM-based machine types
+
+Google Cloud offers machine types based on Intel, AMD and ARM processors. ARM architecture is supported by FortiGate since version 7.2.4 and can be deployed using this module.
+
+***NOTE:*** *make sure T2A family machine types are available in your region*
+
+## Configuration
+
+Deploying an ARM-based instance requires
+- boot image supporting ARM architecture
+- machine type from T2A family
+- using GVNIC network driver (VIRTIO is not supported by T2A machine type family)
+
+Above requirements can be configured by passing the following variables to the module:
+
+```
+image_family  = "fortigate-arm64-72-payg"
+machine_type  = "t2a-standard-4"
+nic_type      = "GVNIC"
+```
+
+or when it's desired to deploy a particular version instead of the latest one:
+
+```
+image_name    = "fortinet-fgt-arm64-724-20230216-001-w-license"
+machine_type  = "t2a-standard-4"
+nic_type      = "GVNIC"
+```

examples/arm-based-machine-type/main.tf

@@ -0,0 +1,16 @@
+module "fgt_ha" {
+  source        = "git::github.com/40net-cloud/fortigate-gcp-ha-ap-lb-terraform?ref=v1.0.0"
+
+  image_family  = "fortigate-arm64-72-payg"
+  machine_type  = "t2a-standard-4"
+  nic_type      = "GVNIC"
+
+  prefix        = "fgt-example-arm"
+  region        = "us-central1"
+  subnets       = [ var.subnet_external, var.subnet_internal, var.subnet_hasync, var.subnet_mgmt]
+  frontends     = ["app1"]
+}
+
+output outputs {
+  value = module.fgt_ha
+}