Skip to content

Terraform module to create Amazon Elasticsearch Service clusters, following the Well-Architected Framework and best AWS practices.

License

Notifications You must be signed in to change notification settings

lgallard/terraform-aws-elasticsearch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Terraform

terraform-aws-elasticsearch

Terraform module to create Amazon Elasticsearch Service clusters, following the Well-Architected Framework and best AWS practices.

Amazon Elasticsearch Service is a fully managed service that makes it easy to deploy, operate, and scale Elasticsearch clusters in the AWS Cloud. Elasticsearch is a popular open-source search and analytics engine for use cases such as log analytics, real-time application monitoring, and clickstream analysis. With Amazon ES, you get direct access to the Elasticsearch APIs; existing code and applications work seamlessly with the service.

Examples

Check the examples folder where you can see how to configure a public ES cluster, and another example showing how to set it with VPC options.

Usage

You can use this module to create your Amazon ES cluster by defining each parameters blocks as follows:

module "aws_es" {

  source  = "lgallard/elasticsearch/aws"

  domain_name           = "elasticsearch_public"
  elasticsearch_version = "7.1"

  cluster_config = {
    dedicated_master_enabled = true
    instance_count           = 3
    instance_type            = "r5.large.elasticsearch"
    zone_awareness_enabled   = true
    availability_zone_count  = 3
  }

  ebs_options = {
    ebs_enabled = "true"
    volume_size = "25"
  }

  encrypt_at_rest = {
    enabled    = true
    kms_key_id = "arn:aws:kms:us-east-1:123456789101:key/cccc103b-4ba3-5993-6fc7-b7e538b25fd8"
  }

  log_publishing_options = {
    index_slow_logs = {
      enabled                          = true
      cloudwatch_log_group_arn         = "arn:aws:logs:us-east-1:123456789101:log-group:/aws/elasticsearch/index_slow_logs:*"
      log_publishing_options_retention = 90
    }
    search_slow_logs = {
      enabled                  = true
      cloudwatch_log_group_arn = "arn:aws:logs:us-east-1:123456789101:log-group:/aws/elasticsearch/search_slow_logs:*"
    }
    es_application_logs = {
      enabled                   = true
      cloudwatch_log_group_name = "es_application_logs_dev"
    }
    audit_logs = {
      enabled                   = false
      cloudwatch_log_group_name = "audit_logs_dev"
    }
  }

  advanced_options = {
    "rest.action.multi.allow_explicit_index" = true
  }

  domain_endpoint_options = {
    enforce_https                   = true
    custom_endpoint_enabled         = true
    custom_endpoint                 = "lgallardo.com"
    custom_endpoint_certificate_arn = "arn:aws:acm:us-east-1:123456789101:certificate/abcd1234-ef11-abcd-1234-abcd1234efef"
  }

  node_to_node_encryption_enabled                = true
  snapshot_options_automated_snapshot_start_hour = 23

  tags = {
    Owner = "sysops"
    env   = "dev"
  }

Note: You can also define the above ElasticSearch cluster using just the module variables. Instead of defining a cluster_config block (list of map), you can set each of the cluster_config_* variables, as shown below:

module "aws_es" {

  source  = "lgallard/elasticsearch/aws"

  domain_name           = "elasticsearch_public"
  elasticsearch_version = "7.1"

  cluster_config_dedicated_master_enabled = true
  cluster_config_instance_count           = 3
  cluster_config_instance_type            = "r5.large.elasticsearch"
  cluster_config_zone_awareness_enabled   = "true"
  cluster_config_availability_zone_count  = 3

  ebs_options_ebs_enabled = true
  ebs_options_volume_size = 25

  encrypt_at_rest_enabled    = true
  encrypt_at_rest_kms_key_id = "alias/aws/es"

  log_publishing_options_enabled  = true
  log_publishing_options_log_type = "INDEX_SLOW_LOGS"

  advanced_options = {
    "rest.action.multi.allow_explicit_index" = true
  }

  node_to_node_encryption_enabled                = true
  snapshot_options_automated_snapshot_start_hour = 23

  tags = {
    Owner = "sysops"
    env   = "dev"
  }

Requirements

Name Version
terraform >= 0.12.9
aws >= 3.35.0
random >=3.1.2

Providers

Name Version
aws 4.18.0
random 3.3.1

Modules

No modules.

Resources

Name Type
aws_cloudwatch_log_group.es_cloudwatch_log_group resource
aws_cloudwatch_log_resource_policy.es_aws_cloudwatch_log_resource_policy resource
aws_elasticsearch_domain.es_domain resource
aws_iam_service_linked_role.es resource
random_password.master_password resource
aws_kms_key.aws_es data source

Inputs

Name Description Type Default Required
access_policies IAM policy document specifying the access policies for the domain string "" no
advanced_options Key-value string pairs to specify advanced configuration options. Note that the values for these configuration options must be strings (wrapped in quotes) or they may be wrong and cause a perpetual diff, causing Terraform to want to recreate your Elasticsearch domain on every apply map(string) {} no
advanced_security_options Options for fine-grained access control any {} no
advanced_security_options_create_random_master_password Whether to create random master password for Elasticsearch master user bool false no
advanced_security_options_enabled Whether advanced security is enabled (Forces new resource) bool false no
advanced_security_options_internal_user_database_enabled Whether the internal user database is enabled. If not set, defaults to false by the AWS API. bool false no
advanced_security_options_master_user_arn ARN for the master user. Only specify if internal_user_database_enabled is not set or set to false) string null no
advanced_security_options_master_user_password The master user's password, which is stored in the Amazon Elasticsearch Service domain's internal database. Only specify if internal_user_database_enabled is set to true. string null no
advanced_security_options_master_user_username The master user's username, which is stored in the Amazon Elasticsearch Service domain's internal database. Only specify if internal_user_database_enabled is set to true. string null no
advanced_security_options_random_master_password_length Length of random master password to create number 16 no
cloudwatch_log_enabled Change to false to avoid deploying any Cloudwatch Logs resources bool true no
cluster_config Cluster configuration of the domain any {} no
cluster_config_availability_zone_count Number of Availability Zones for the domain to use with number 3 no
cluster_config_cold_storage_options_enabled Indicates whether to enable cold storage for an Elasticsearch domain bool false no
cluster_config_dedicated_master_count Number of dedicated master nodes in the cluster number 3 no
cluster_config_dedicated_master_enabled Indicates whether dedicated master nodes are enabled for the cluster bool true no
cluster_config_dedicated_master_type Instance type of the dedicated master nodes in the cluster string "r5.large.elasticsearch" no
cluster_config_instance_count Number of instances in the cluster number 3 no
cluster_config_instance_type Instance type of data nodes in the cluster string "r5.large.elasticsearch" no
cluster_config_warm_count The number of warm nodes in the cluster number null no
cluster_config_warm_enabled Indicates whether to enable warm storage bool false no
cluster_config_warm_type The instance type for the Elasticsearch cluster's warm nodes string null no
cluster_config_zone_awareness_enabled Indicates whether zone awareness is enabled. To enable awareness with three Availability Zones bool false no
cognito_options Options for Amazon Cognito Authentication for Kibana any {} no
cognito_options_enabled Specifies whether Amazon Cognito authentication with Kibana is enabled or not bool false no
cognito_options_identity_pool_id ID of the Cognito Identity Pool to use string "" no
cognito_options_role_arn ARN of the IAM role that has the AmazonESCognitoAccess policy attached string "" no
cognito_options_user_pool_id ID of the Cognito User Pool to use string "" no
create_service_link_role Create service link role for AWS Elasticsearch Service bool true no
domain_endpoint_options Domain endpoint HTTP(S) related options. any {} no
domain_endpoint_options_custom_endpoint Fully qualified domain for your custom endpoint string null no
domain_endpoint_options_custom_endpoint_certificate_arn ACM certificate ARN for your custom endpoint string null no
domain_endpoint_options_custom_endpoint_enabled Whether to enable custom endpoint for the Elasticsearch domain bool false no
domain_endpoint_options_enforce_https Whether or not to require HTTPS bool false no
domain_endpoint_options_tls_security_policy The name of the TLS security policy that needs to be applied to the HTTPS endpoint. Valid values: Policy-Min-TLS-1-0-2019-07 and Policy-Min-TLS-1-2-2019-07 string "Policy-Min-TLS-1-2-2019-07" no
domain_name Name of the domain string n/a yes
ebs_enabled Whether EBS volumes are attached to data nodes in the domain bool true no
ebs_options EBS related options, may be required based on chosen instance size any {} no
ebs_options_iops The baseline input/output (I/O) performance of EBS volumes attached to data nodes. Applicable only for the Provisioned IOPS EBS volume type number 0 no
ebs_options_volume_size The size of EBS volumes attached to data nodes (in GB). Required if ebs_enabled is set to true number 10 no
ebs_options_volume_type The type of EBS volumes attached to data nodes string "gp2" no
elasticsearch_version The version of Elasticsearch to deploy. string "7.1" no
enabled Change to false to avoid deploying any AWS ElasticSearch resources bool true no
encrypt_at_rest Encrypt at rest options. Only available for certain instance types any {} no
encrypt_at_rest_enabled Whether to enable encryption at rest bool true no
encrypt_at_rest_kms_key_id The KMS key id to encrypt the Elasticsearch domain with. If not specified then it defaults to using the aws/es service KMS key string "alias/aws/es" no
log_publishing_options Options for publishing slow logs to CloudWatch Logs any {} no
log_publishing_options_retention Retention in days for the created Cloudwatch log group number 90 no
node_to_node_encryption Node-to-node encryption options any {} no
node_to_node_encryption_enabled Whether to enable node-to-node encryption bool true no
snapshot_options Snapshot related options any {} no
snapshot_options_automated_snapshot_start_hour Hour during which the service takes an automated daily snapshot of the indices in the domain number 0 no
tags A mapping of tags to assign to the resource map(any) {} no
timeouts Timeouts map. map(any) {} no
timeouts_update How long to wait for updates. string null no
vpc_options VPC related options, see below. Adding or removing this configuration forces a new resource any {} no
vpc_options_security_group_ids List of VPC Security Group IDs to be applied to the Elasticsearch domain endpoints. If omitted, the default Security Group for the VPC will be used list(any) [] no
vpc_options_subnet_ids List of VPC Subnet IDs for the Elasticsearch domain endpoints to be created in list(any) [] no

Outputs

Name Description
arn Amazon Resource Name (ARN) of the domain
domain_id Unique identifier for the domain
endpoint Domain-specific endpoint used to submit index, search, and data upload requests
kibana_endpoint Domain-specific endpoint for kibana without https scheme
master_password Master password
master_username Master username
vpc_options_availability_zones If the domain was created inside a VPC, the names of the availability zones the configured subnet_ids were created inside
vpc_options_vpc_id If the domain was created inside a VPC, the ID of the VPC