spacelift-automation

This Terraform child module provides infrastructure automation for projects in Spacelift.

Overview

The spacelift-automation root module is designed to streamline the deployment and management of all Spacelift infrastructure, including creating a Spacelift Stack to manage itself.

It automates the creation of "child" stacks and all the required accompanying Spacelift resources. For each enabled root module it creates:

1. Spacelift Stack You can think about a stack as a combination of source code, state file and configuration in the form of environment variables and mounted files.
2. Spacelift Stack Destructor Required to destroy the resources of a Stack before deleting it. Destroying this resource will delete the resources in the stack. If this resource needs to be deleted and the resources in the stacks are to be preserved, ensure that the deactivated attribute is set to true.
3. Spacelift AWS Integration Attachment Associates a specific AWS IAM role with a stack to allow it to assume that role. The IAM role typically has permissions to manage specific AWS resources, and Spacelift assumes this role to run the operations required by the stack.
4. Spacelift Initialization Hook Prepares your environment before executing infrastructure code. This custom script copies corresponding Terraform tfvars files into a working directory before any Spacelift run or task as a spacelift.auto.tfvars file. This ensures your tfvars are automatically loaded into the OpenTofu/Terraform execution environment.

Usage

Spacelift Automation logic is opinionated and heavily relies on certain repository structures. This module is configured to track all the files in the given root module directory and create Spacelift Stacks based on the provided configuration.

We support the following root module directory structures, which are controlled by the var.root_modules_structure variable:

MultiInstance (the default)

This is the default structure that we expect and recommend. This is intended for root modules that manage multiple state files (instances) through workspaces or Dynamic Backend configurations.

Structure requirements:

• Stack configs are placed in <root_modules_path>/<root_module>/stacks directory for each workspace / instance of that stack. e.g. root-modules/k8s-cluster/stacks/dev.yaml and root-modules/k8s-cluster/stacks/stage.yaml
• Terraform variables are placed in <root_modules_path>/<root_module>/tfvars directory for each workspace / instance of that stack. e.g. root-modules/k8s-cluster/tfvars/dev.tfvars and root-modules/k8s-cluster/tfvars/stage.tfvars
• Stack config files and tfvars files must be equal to OpenTofu/Terraform workspace, e.g. stacks/dev.yaml and tfvars/dev.tfvars for a workspace named dev.
• Common configs are placed in <root_modules_path>/<root_module>/stacks/common.yaml file (or var.common_config_file value). This is useful when you know that some values should be shared across all the stacks created for a root module. For example, all stacks that manage Spacelift Policies must use the administrative: true setting or all stacks must share the same labels.

We have an example of this structure in the examples/complete, which looks like the following:

├── root-modules
│   ├── spacelift-aws-role
│   │   ├── stacks
│   │   │   └── dev.yaml
│   │   │   └── stage.yaml
│   │   │   └── common.yaml
│   │   ├── tfvars
│   │   │   └── dev.tfvars
│   │   │   └── stage.tfvars
│   │   ├── variables.tf
│   │   ├── main.tf
│   │   └── versions.tf
│   ├── k8s-cluster
│   │   ├── stacks
│   │   │   └── dev.yaml
│   │   │   └── prod.yaml
│   │   │   └── common.yaml
│   │   ├── tfvars
│   │   │   └── dev.tfvars
│   │   │   └── prod.tfvars
│   │   ├── variables.tf
│   │   ├── main.tf
│   │   └── versions.tf
...

The spacelift-automation/main.tf file looks something like this:

github_enterprise = {
  namespace = "masterpointio"
}
repository = "terraform-spacelift-automation"

# Stacks configurations
root_modules_path        = "root-modules"
all_root_modules_enabled = true

aws_integration_id = "ZDPP8SKNVG0G27T4"

The configuration above creates the following stacks:

• spacelift-aws-role-dev
• spacelift-aws-role-stage
• k8s-cluster-dev
• k8s-cluster-prod

These stacks have the following configuration:

• Stacks track changes in GitHub repo github.com/masterpointio/terraform-spacelift-automation, branch main (the default), and directrory root-modules.
• Common configuration is defined in root-modules/spacelift-aws-role/stacks/common.yaml and applied to both Stacks. However, if there is an override in a Stack config (e.g. root-modules/spacelift-aws-role/stacks/dev.yaml), it takes precedence over common configs.
• Corresponding Terraform variables are generated by an Initialization Hook and placed in the root of each Stack's working directory during each run or task. For example, the content of the file root-modules/spacelift-aws-role/tfvars/dev.tfvars will be copied to working directory of the Stack spacelift-aws-role-dev as file spacelift.auto.tfvars allowing the OpenTofu/Terraform inputs to be automatically loaded.
  • If you would like to disable this functionality, you can set tfvars.enabled in the Stack's YAML file to false.

SingleInstance

This is a special case where each root module directory only manages one state file (instance). Each time you want to create a new instance of a root module, you need to create a new directory with the same code and change your inputs. We do not recommend this structure as it is less flexible and easily leads to anti-patterns, but it is supported.

Structure requirements:

• Stack configs are placed in <root_modules_path>/<root_module>/stack.yaml directory. e.g. root-modules/rds-cluster/stack.yaml
• Tfvars values are not supported in this structure. In this structure, we suggest you just add your tfvars as ***.auto.tfvars or hardcode your values directly in root module code.

Here is an example of this structure that we have in the examples/single-instance directory:

├── root-modules
│   ├── spacelift-automation
│   │   ├── stack.yaml
│   │   ├── variables.tf
│   │   ├── main.tf
│   │   └── versions.tf
│   ├── rds-cluster-dev
│   │   ├── stack.yaml
│   │   ├── main.tf
│   │   └── versions.tf
│   ├── rds-cluster-prod
│   │   ├── stack.yaml
│   │   ├── main.tf
│   │   └── versions.tf
│   ├── random-pet
│   │   ├── stack.yaml
│   │   ├── variables.tf
│   │   ├── main.tf
│   │   └── versions.tf
...

The configuration above creates the following Spacelift Stacks:

• spacelift-automation
• rds-cluster-dev
• rds-cluster-prod
• random-pet

These stacks will be configured using the settings in the stack.yaml file.

FAQs

Can I create a Spacelift Stack for Spacelift Automation? (Recommended)

Spacelift Automation can manage itself as a Stack as well, and we recommend this so you can fully automate your Stack management upon merging to your given branch. Follow these steps to achieve that:

1. Create a new vanilla OpenTofu/Terraform root module in <root_modules_path>/spacelift-automation that consumes this child module and supplies the necessary configuration for your unique setup. e.g.

   # root-modules/spacelift-automation/main.tf
   
   module "spacelift-automation" {
     source  = "masterpointio/automation/spacelift"
     version = "x.x.x" # Always pin a version, use the latest version from the release page.
   
     # GitHub configuration
     github_enterprise = {
       namespace = "masterpointio"
     }
     repository = "your-infrastructure-repo"
   
     # Stacks configurations
     root_modules_path        = "../../root-modules"
     all_root_modules_enabled = true
   
     aws_integration_id = "ZDPP8SKNVG0G27T4"
   }

2. Optionally, create a Terraform workspace that will be used for your Automation configuration, e.g.:

   tofu workspace new main

   Remember that Stack config and tfvars file name must be equal to the workspace e.g. main.yaml and main.tfvars. If you choose not to create a new workspace, this can be default.yaml and default.tfvars.

3. Apply the spacelift-automation root module.

4. Move the Automation configs to the <root-modules>/spacelift-automation/stacks directory and push the changes to the tracked repo and branch.

5. After pushed to your repo's tracked branch, Spacelift Automation will track the addition of new root modules and create Stacks for them.

Check out an example configuration in the examples/complete.

What goes in a Stack config file? e.g. stacks/dev.yaml, stacks/common.yaml, stack.yaml, etc

Most settings that you would set on the Spacelift Stack resource are supported. Additionally, you can include certain Stack specific settings that will override this module's defaults like default_tf_workspace_enabled, tfvars.enabled, and similar. See the code for full details.

Why are variable values provided separately in tfvars/ and not in the yaml file?

This is to support easy local and outside-spacelift operations. Keeping variable values in a tfvars file per workspace allows you to simply pass that file to the relevant CLI command locally via the -var-file option so that you don't need to provide values individually. e.g. tofu plan -var-file=tfvars/dev.tfvars

Requirements

Name	Version
terraform	>= 1.6
spacelift	>= 1.14

Providers

Name	Version
spacelift	1.19.0

Modules

Name	Source	Version
deep	cloudposse/config/yaml//modules/deepmerge	1.0.2

Resources

Name	Type
spacelift_aws_integration_attachment.default	resource
spacelift_drift_detection.default	resource
spacelift_stack.default	resource
spacelift_stack_destructor.default	resource

Inputs

Name	Description	Type	Default	Required
administrative	Flag to mark the stack as administrative	bool	false	no
after_apply	List of after-apply scripts	list(string)	[]	no
after_destroy	List of after-destroy scripts	list(string)	[]	no
after_init	List of after-init scripts	list(string)	[]	no
after_perform	List of after-perform scripts	list(string)	[]	no
after_plan	List of after-plan scripts	list(string)	[]	no
all_root_modules_enabled	When set to true, all subdirectories in root_modules_path will be treated as root modules.	bool	false	no
autodeploy	Flag to enable/disable automatic deployment of the stack	bool	true	no
autoretry	Flag to enable/disable automatic retry of the stack	bool	false	no
aws_integration_attachment_read	Indicates whether this attachment is used for read operations.	bool	true	no
aws_integration_attachment_write	Indicates whether this attachment is used for write operations.	bool	true	no
aws_integration_enabled	Indicates whether the AWS integration is enabled.	bool	false	no
aws_integration_id	ID of the AWS integration to attach.	string	null	no
before_apply	List of before-apply scripts	list(string)	[]	no
before_destroy	List of before-destroy scripts	list(string)	[]	no
before_init	List of before-init scripts	list(string)	[]	no
before_perform	List of before-perform scripts	list(string)	[]	no
before_plan	List of before-plan scripts	list(string)	[]	no
branch	Specify which branch to use within the infrastructure repository.	string	"main"	no
common_config_file	Name of the common configuration file for the stack across a root module.	string	"common.yaml"	no
default_tf_workspace_enabled	Enables the use of default Terraform workspace instead of managing multiple workspaces within a root module. NOTE: We encourage the use of Terraform workspaces to manage multiple environments. However, you will want to disable this behavior if you're utilizing different backends for each instance of your root modules (we call this "Dynamic Backends").	bool	false	no
description	Description of the stack	string	"Managed by spacelift-automation Terraform root module."	no
destructor_enabled	Flag to enable/disable the destructor for the Stack.	bool	false	no
drift_detection_enabled	Flag to enable/disable Drift Detection configuration for a Stack.	bool	false	no
drift_detection_ignore_state	Controls whether drift detection should be performed on a stack in any final state instead of just 'Finished'.	bool	false	no
drift_detection_reconcile	Flag to enable/disable automatic reconciliation of drifts.	bool	false	no
drift_detection_schedule	The schedule for drift detection.	list(string)	[ "0 4 * * *" ]	no
drift_detection_timezone	The timezone for drift detection.	string	"UTC"	no
enable_local_preview	Indicates whether local preview runs can be triggered on this Stack.	bool	false	no
enable_well_known_secret_masking	Indicates whether well-known secret masking is enabled.	bool	true	no
enabled_root_modules	List of root modules where to look for stack config files. Ignored when all_root_modules_enabled is true. Example: ["spacelift-automation", "k8s-cluster"]	list(string)	[]	no
github_action_deploy	Indicates whether GitHub users can deploy from the Checks API.	bool	true	no
github_enterprise	The GitHub VCS settings	object({ namespace = string id = optional(string) })	n/a	yes
manage_state	Determines if Spacelift should manage state for this stack.	bool	false	no
protect_from_deletion	Protect this stack from accidental deletion. If set, attempts to delete this stack will fail.	bool	false	no
repository	The name of your infrastructure repo	string	n/a	yes
root_modules_path	The path, relative to the root of the repository, where the root module can be found.	string	"root-modules"	no
space_id	Place the stack in the specified space_id.	string	"root"	no
terraform_smart_sanitization	Indicates whether runs on this will use terraform's sensitive value system to sanitize the outputs of Terraform state and plans in spacelift instead of sanitizing all fields.	bool	false	no
terraform_version	Terraform version to use.	string	"1.7.2"	no
terraform_workflow_tool	Defines the tool that will be used to execute the workflow. This can be one of OPEN_TOFU, TERRAFORM_FOSS or CUSTOM.	string	"OPEN_TOFU"	no
worker_pool_id	ID of the worker pool to use. NOTE: worker_pool_id is required when using a self-hosted instance of Spacelift.	string	null	no

Outputs

Name	Description
spacelift_stacks	A map of Spacelift stacks with selected attributes. To reduce the risk of accidentally exporting sensitive data, only a subset of attributes is exported.

Contributing

Contributions are welcome and appreciated!

Found an issue or want to request a feature? Open an issue

Want to fix a bug you found or add some functionality? Fork, clone, commit, push, and PR and we'll check it out.

If you have any issues or are waiting a long time for a PR to get merged then feel free to ping us at [email protected].

Built By