Comet Infrastructure Deployment: Terraform on AWS

Terraform module for deploying infrastructure components to run CometML.

Deployment

Prerequisites:

• Git installed
• AWS CLI installed
  • Access to an AWS account and credentials that allow you to create resources
  • Ensure AWS_ACCESS_KEY_ID and AWS_SECRET_KEY_ID environment variables are set in your session
• Terraform installed

Infrastructure Deployment:

• Follow the steps below to deploy directly from the GitHub repository.
  • Clone the repository to your local machine: git clone https://github.com/comet-ml/terraform_aws_comet.git
  • Move into the deployment directory: cd terraform-aws-comet
  • Initialize the directory: terraform init
  • Within terraform.tfvars, set your module toggles to enable the desired infrastructure components and set any related inputs
  • Provision the resources: terraform apply

A note on state management:

• This configuration stores the Terraform state locally by default. To store the state file remotely in S3, a backend block can be nested within the terraform block inside versions.tf if applying directly from this configuration, or within your terraform block if calling the module. Below is an example of such a configuration:

terraform {
  backend "s3" {
    bucket = "mybucket"
    key    = "path/to/my/key"
    region = "us-east-1"
  }
}

• More on state management in S3 can be found here

Requirements

Name	Version
terraform	>= 1.0
aws	~> 5.1
helm	~> 2.10
kubernetes	~> 2.21

Providers

Name	Version
aws	5.8.0

Modules

Name	Source	Version
comet_ec2	./modules/comet_ec2	n/a
comet_ec2_alb	./modules/comet_ec2_alb	n/a
comet_eks	./modules/comet_eks	n/a
comet_elasticache	./modules/comet_elasticache	n/a
comet_rds	./modules/comet_rds	n/a
comet_s3	./modules/comet_s3	n/a
comet_vpc	./modules/comet_vpc	n/a

Resources

Name	Type
aws_eks_cluster_auth.this	data source

Inputs

Name	Description	Type	Default	Required
availability_zones	List of availability zones from region	list(string)	null	no
comet_ec2_ami_type	Operating system type for the EC2 instance AMI	string	"ubuntu22"	no
comet_ec2_instance_count	Number of EC2 instances to provision	number	1	no
comet_ec2_instance_type	Instance type for the EC2 instance	string	"m5.4xlarge"	no
comet_ec2_key	Name of the SSH key to configure on the EC2 instance	string	null	no
comet_ec2_volume_size	Size, in gibibytes (GiB), for the EC2 instance root volume	number	1024	no
comet_ec2_volume_type	EBS volume type for the EC2 instance root volume	string	"gp2"	no
comet_private_subnets	List of private subnets IDs from existing VPC to provision resources in	list(string)	null	no
comet_public_subnets	List of public subnets IDs from existing VPC to provision resources in	list(string)	null	no
comet_vpc_id	ID of an existing VPC to provision resources in	string	null	no
eks_aws_cloudwatch_metrics	Enables AWS Cloudwatch Metrics in the EKS cluster	bool	true	no
eks_aws_load_balancer_controller	Enables the AWS Load Balancer Controller in the EKS cluster	bool	true	no
eks_cert_manager	Enables cert-manager in the EKS cluster	bool	false	no
eks_cluster_name	Name for EKS cluster	string	"comet-eks"	no
eks_cluster_version	Kubernetes version of the EKS cluster	string	"1.27"	no
eks_external_dns	Enables ExternalDNS in the EKS cluster	bool	false	no
eks_external_dns_r53_zones	Route 53 zones for external-dns to have access to	list(string)	[ "arn:aws:route53:::hostedzone/XYZ" ]	no
eks_mng_ami_type	AMI family to use for the EKS nodes	string	"AL2_x86_64"	no
eks_mng_desired_size	Desired number of nodes in EKS cluster	number	3	no
eks_mng_disk_size	Size of the storage disks for nodes in EKS cluster	number	500	no
eks_mng_max_size	Maximum number of nodes in EKS cluster	number	6	no
eks_mng_name	Name for the EKS managed nodegroup	string	"mng"	no
eks_node_types	Node instance types for EKS managed node group	list(string)	[ "m5.4xlarge" ]	no
elasticache_allow_from_sg	Security group from which to allow connections to ElastiCache, to use when provisioning with existing compute	string	null	no
elasticache_auth_token	Auth token for ElastiCache	string	null	no
elasticache_engine	Engine type for ElastiCache cluster	string	"redis"	no
elasticache_engine_version	Version number for ElastiCache engine	string	"5.0.6"	no
elasticache_instance_type	ElastiCache instance type	string	"cache.r4.xlarge"	no
elasticache_num_cache_nodes	Number of nodes in the ElastiCache cluster	number	1	no
elasticache_param_group_name	Name for the ElastiCache cluster parameter group	string	"default.redis5.0"	no
elasticache_transit_encryption	Enable transit encryption for ElastiCache	bool	false	no
enable_ec2	Toggles the comet_ec2 module, to provision EC2 resources for running Comet	bool	n/a	yes
enable_ec2_alb	Toggles the comet_ec2_alb module, to provision an ALB in front of the EC2 instance	bool	n/a	yes
enable_eks	Toggles the comet_eks module, to provision EKS resources for running Comet	bool	n/a	yes
enable_elasticache	Toggles the comet_elasticache module for provisioning Comet Redis on elasticache	bool	n/a	yes
enable_rds	Toggles the comet_rds module for provisioning Comet RDS database	bool	n/a	yes
enable_s3	Toggles the comet_s3 module for provisioning Comet S3 bucket	bool	n/a	yes
enable_vpc	Toggles the comet_vpc module, to provision a new VPC for hosting the Comet resources	bool	n/a	yes
environment	Deployment environment, i.e. dev/stage/prod, etc	string	"dev"	no
rds_allow_from_sg	Security group from which to allow connections to RDS, to use when provisioning with existing compute	string	null	no
rds_backup_retention_period	Days specified for RDS snapshotretention period	number	7	no
rds_database_name	Name for the application database in RDS	string	"logger"	no
rds_engine	Engine type for RDS database	string	"aurora-mysql"	no
rds_engine_version	Engine version number for RDS database	string	"5.7"	no
rds_iam_db_auth	Enables IAM auth for the database in RDS	bool	true	no
rds_instance_count	Number of RDS instances in the database cluster	number	2	no
rds_instance_type	Instance type for RDS database	string	"db.r5.xlarge"	no
rds_preferred_backup_window	Backup window for RDS	string	"07:00-09:00"	no
rds_root_password	Root password for RDS database	string	n/a	yes
rds_storage_encrypted	Enables encryption for RDS storage	bool	true	no
region	AWS region to provision resources in	string	n/a	yes
s3_bucket_name	Name for S3 bucket	string	n/a	yes
single_nat_gateway	Controls whether single NAT gateway used for all public subnets	bool	true	no
ssl_certificate_arn	ARN of the ACM certificate to use for the ALB	string	null	no

Outputs

Name	Description
comet_alb_dns_name	DNS name of the ALB fronting the Comet EC2 instance
comet_ec2_instance	ID of the Comet EC2 instance
comet_ec2_public_ip	EIP associated with the Comet EC2 instance
configure_kubectl	Configure kubectl: run the following command to update your kubeconfig with the newly provisioned cluster.
mysql_host	Endpoint for the RDS instance
region	Region resources are provisioned in