Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow es domain attributes to be parameterized #61

Open
fredliporace opened this issue May 5, 2021 · 0 comments
Open

Allow es domain attributes to be parameterized #61

fredliporace opened this issue May 5, 2021 · 0 comments

Comments

@fredliporace
Copy link
Owner

fredliporace commented May 5, 2021

ES domain creation

Possible difficulty is the use of CDK types, elasticsearch.ElasticsearchVersion.V7_7 for instance. But some useful parameters may be directly defined by **args, such as volume_size, instance_type, etc.

        esd = elasticsearch.Domain(
            self,
            id="cbers2stac",
            # This is the version currently used by localstack
            version=elasticsearch.ElasticsearchVersion.V7_7,
            ebs=elasticsearch.EbsOptions(enabled=True, volume_size=10),
            capacity=elasticsearch.CapacityConfig(
                data_node_instance_type="t2.small.elasticsearch", data_nodes=1,
            ),
        )

Lambda to insert into ES, check environment vars:

            self.create_lambda(
                id="insert_into_elastic_lambda",
                code=aws_lambda.Code.from_asset(path="cbers2stac/elasticsearch"),
                handler="es.create_documents_handler",
                runtime=aws_lambda.Runtime.PYTHON_3_7,
                environment={**self.lambdas_env_,
                             **{"ES_STRIPPED": "YES",
                                "BULK_CALLS": "1",
                                "BULK_SIZE": "10"}},
                layers=[self.layers_["common_layer"]],
                timeout=core.Duration.seconds(30),
                dead_letter_queue=self.queues_["dead_letter_queue"],
                # Concurrent executions tuned to work with t2.small.elasticsearch
                reserved_concurrent_executions=5,
                description="Consume STAC items from queue, inserting into ES",
            )
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant