SDC AWS Pipeline Architecture

This repository contains Terraform configurations for managing AWS infrastructure across multiple missions, with separate environments for development and production.

Repository Structure

.
├── base-infrastructure-terraform/    # Base shared infrastructure
└── pipeline-infrastructure-terraform/  # Mission-specific pipeline infrastructure

Infrastructure Management

Base Infrastructure

• Managed in base-infrastructure-terraform/
• Uses default Terraform workspace
• Contains shared resources used across all pipelines
• Key components:
  • Base AWS infrastructure setup
  • Executor Lambda function
  • Common configuration in swxsoc.auto.tfvars

Pipeline Infrastructure

• Managed in pipeline-infrastructure-terraform/
• Uses separate workspaces for each mission's environments
• Two workspaces per mission: development (dev) and production (prod)
• Components:
  • Mission-specific configurations (hermes.tfvars, padre.tfvars)
  • Lambda functions for artifacts, processing, and sorting
  • Pipeline-specific infrastructure

Getting Started

1. Install prerequisites:

   • Terraform
   • AWS CLI configured with appropriate credentials

2. Base Infrastructure Deployment:

cd base-infrastructure-terraform
terraform init
terraform plan
terraform apply

3. Pipeline Infrastructure Deployment:

cd pipeline-infrastructure-terraform
terraform workspace select <environment>-<mission>
terraform init
terraform plan -var-file=<mission>.tfvars
terraform apply -var-file=<mission>.tfvars

Documentation

Comprehensive documentation is available in the docs/ directory:

• Development Guide: docs/dev-guide/
• User Guide: docs/user-guide/
• Pipeline Diagrams: docs/images/

License

See LICENSE file for details.