This project automates the deployment of a backup service that periodically copies data from a source S3 bucket to a backup S3 bucket on AWS. It uses AWS Lambda for processing, ECR for container image storage, and Terraform for infrastructure as code.
- Automated Backup: Configurable to run at a scheduled time.
- Serverless Architecture: Utilizes AWS Lambda for efficient, event-driven processing.
- Containerized Application: Dockerized Python application for easy deployment and isolation.
- Infrastructure as Code: Managed through Terraform, ensuring consistent and repeatable setups.
- AWS CLI, configured with appropriate access
- Docker
- Terraform
/terraform
: Contains Terraform configurations split into two directories:/ecr
: For ECR repository creation./lambda
: For Lambda function and related resources.
/config
: Contains the YAML configuration file./src
: Contains the application's source code.Dockerfile
: Docker configuration for the Lambda function.main.py
: Python script executed by the Lambda function.Makefile
: Contains commands to facilitate building, deploying, and destroying the infrastructure.README.md
: Documentation for the project.
- Configure AWS CLI: Make sure AWS CLI is installed and configured with the necessary access permissions.
- Define Environment Variables:
- Copy
.env.example
to.env
and fill in the required AWS and application-specific variables.
- Copy
- Deploy Infrastructure:
- Run
make deploy
to deploy the ECR repository, push the Docker image, and set up the Lambda function along with its necessary resources.
- Run
- Verify Deployment:
- Check the AWS Management Console to ensure that resources are deployed correctly.
- The Lambda function is triggered as per the schedule defined in the CloudWatch Event Rule (default is every day at 3 AM UTC).
- The function copies data from the specified source S3 bucket to the backup S3 bucket.
build
: Builds the Docker image.deploy
: Deploys the entire infrastructure and the application.destroy
: Removes all deployed AWS resources.
- Variables are defined in
variables.tf
files within each Terraform directory. - The ECR and Lambda resources are managed separately to allow for independent updates.
- Lambda Errors: Check the AWS Lambda console for logs and error messages.
- Terraform Issues: Run
terraform plan
to identify configuration mismatches.
- Integrate CI/CD for automated builds and deployments.
- Implement monitoring and alerting for backup failures.