Skip to content

Latest commit

 

History

History
309 lines (238 loc) · 20.8 KB

README.md

File metadata and controls

309 lines (238 loc) · 20.8 KB
Raw

Disconnected OpenShift Testbed

A terraform module for creating a testbed for Disconnected OpenShift clusters. It configures resources on Amazon to emulate a disconnected environment, providing the means for a user to practice or demonstrate "sneakernet" use-cases for getting OpenShift content into a disconnected environment on AWS, while still allowing for API connectivity. This is similar enough to other IPI use cases (Metal3, vSphere) that are very common on other infrastructures, as well as closely related to the experience on "disconnected" AWS overlays, to be a meaningful test bench.

For an example of this Terraform module's intended use cases, including more robust configuration post-provisioning please see the oc-mirror E2E testing framework repository.

Basic Information

Environment Prerequisites

  • Terraform >= 1.4.2
  • An AWS profile defined in ~/.aws/credentials or IAM credentials (access key id and secret is fine) exported
    • This AWS profile needs to have a significant amount of privilege
    • You should have a Route53 Hosted Zone available on the account

What do I get?

This terraform module creates a VPC with 7 subnets across 3 availability zones:

  • Three of those subnets (one in each zone) are on a traditional AWS IGW that enables outbound internet access
    • These are the "connected" subnets
  • One of those subnets is the designated NAT subnet, with a unique CIDR, access to the IGW, and source/dest check disabled on AWS
  • Three of them (one in each zone) are on an AWS subnet with a route set to the instance in the NAT subnet
    • These are the "isolated" subnets

Three instances are stood up:

  • One in a "connected" subnet, designated to be a registry
    • TCP ports 443 and 80 are allowed to this instance
    • An elastic IP and Route53 record are created for this instance
    • You need to configure this as a registry yourself
  • One in the NAT subnet, designated to be the proxy for isolated subnets
    • An elastic IP and Route53 record are created for this instance
    • You need to configure this as a proxy yourself
  • One in an "isolated" subnet, designated to be the disconnected bastion
    • You need to ensure that proxy configuration meets your expectations for this instance

An S3 bucket and IAM credentials to use it (read/write) are created:

  • These are designed for use by the registry

And finally, a set of IAM credentials which are adequate to install OpenShift are created.

Example usage

Reference the module from your own terraform, with an initialized AWS provider. An example terraform file might look like this:

module "testbed" {
  source = "github.com/jharmison-redhat/disconnected-openshift-testbed"

  public_key     = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIC5Da2XARZmB8KsjASv6MQoAS6sAXrw0yE5Y8ANJ5yTG"
  cluster_name   = "disco"
  cluster_domain = "redhat4govaws.io"
}

provider "aws" {
  region = "us-west-2"
  default_tags {
    tags = {
      Name    = "disco.redhat4govaws.io"
      Project = "disconnected-openshift-testbed"
    }
  }
}

Some more robust examples, including parametrized ones with outputs, are available in the examples directory. Terraform outputs are expected to be required for consumption by follow-on automation.

Module Documentation

Requirements

Name Version
terraform >= 1.4.2
aws 4.58.0

Providers

Name Version
aws 4.58.0

Modules

Name Source Version
ocp_installer ./modules/ocp_installer n/a
registry ./modules/registry n/a
vpc ./modules/vpc n/a

Resources

Name Type
aws_key_pair.ec2_key resource
aws_ami.rhel data source
aws_availability_zones.available data source
aws_region.current data source
aws_route53_zone.public data source

Inputs

Name Description Type Default Required
ami_type The AMI type to use, Access2 or Hourly. Availability may depend on your AWS account being linked with Red Hat Cloud Access. string "Hourly" no
bastion_disk_gb The size of the disk, in GB, for the bastion instance. Expected to be large, to support sneakernetting of content. number 500 no
cluster_domain The name of the domain under which your OpenShift cluster will reside (Note that this needs to be a Hosted Zone managed in Route53). string n/a yes
cluster_name The name you will be giving your OpenShift cluster in metadata.name in install-config.yaml. A new private HostedZone will be created for this as a subdomain of the cluster_domain. string n/a yes
instance_password The password to set for the ec2-user on created instances. string "" no
large_flavor The AWS flavor to use for the larger instance (registry, bastion). string "t3.large" no
proxy_disk_gb The size of the disk, in GB, for the proxy instance. number 20 no
public_key The SSH public key string to use for the instances. string n/a yes
registry_disk_gb The size of the disk, in GB, for the registry instance. Since the registry instance is expected to use S3 storage, can be small. number 20 no
rhel_version The major version of RHEL to use for the AMI selection. string "8" no
small_flavor The AWS flavor to use for smaller instance (proxy). string "t3.small" no

Outputs

Name Description
bastion_instance Information about the bastion instance.
ocp_installer The IAM Access Key ID and Secret for the OpenShift installation user.
private_dns_zone The DNS name of the new private DNS zone, internal to the VPC.
proxy_instance Information about the proxy instance.
registry_bucket The AWS S3 bucket for the registry and IAM credentials required to access it.
registry_instance Information about the registry instance.
vpc Information about the provisioned VPC and its networks.

VPC Submodule Documentation

Requirements

Name Version
terraform >= 1.4.2
aws 4.58.0

Providers

Name Version
aws 4.58.0

Modules

No modules.

Resources

Name Type
aws_default_route_table.public resource
aws_default_security_group.default resource
aws_eip.proxy resource
aws_instance.bastion resource
aws_instance.proxy resource
aws_internet_gateway.default resource
aws_route53_record.bastion_private resource
aws_route53_record.proxy resource
aws_route53_record.proxy_private resource
aws_route53_zone.private resource
aws_route_table.private resource
aws_route_table_association.private resource
aws_subnet.nat resource
aws_subnet.private resource
aws_subnet.public resource
aws_vpc.vpc resource

Inputs

Name Description Type Default Required
ami_id The AMI to use for the proxy and bastion instances. string n/a yes
availability_zones The availability zones to create subnets for. list(string) n/a yes
bastion_disk_gb The size of the disk, in GB, for the bastion instance. Expected to be large, to support sneakernetting of content. number 500 no
bastion_flavor The instance type to use for the isolated bastion host. string "t3.small" no
bastion_hostname The hostname to use when building the bastion instance. string "bastion" no
cluster_domain The name of the domain under which your OpenShift cluster will reside (Note that this needs to be a Hosted Zone managed in Route53). string n/a yes
cluster_name The name you will be giving your OpenShift cluster in metadata.name in install-config.yaml. A new private HostedZone will be created for this as a subdomain of the cluster_domain. string n/a yes
hosted_zone The Route53 Hosted Zone ID for the the cluster_domain, for creating public proxy records. string n/a yes
instance_password The password to set for the ec2-user on the proxy and bastion instances. string "" no
proxy_disk_gb The size of the disk, in GB, for the proxy instance. number 20 no
proxy_flavor The instance type to use for the proxy instance. string "t3.small" no
proxy_hostname The hostname to use when building the proxy instance and creating Route 53 records for it. string "proxy" no
ssh_key The SSH public key to use for the proxy and bastion instances - must already exist as an aws_key_pair! string n/a yes
vpc_cidr The CIDR-notation subnet for the entire VPC. string "10.0.0.0/16" no

Outputs

Name Description
bastion_instance Information about the bastion instance.
private_subnets Details about the subnets that are isolated by routing through the proxy.
private_zone The private Hosted Zone created for the VPC.
proxy_instance Information about the proxy instance.
public_subnets Details about the subnets that route through the IGW to the public internet.

Registry Submodule Documentation

Requirements

Name Version
terraform >= 1.4.2
aws 4.58.0

Providers

Name Version
aws 4.58.0

Modules

No modules.

Resources

Name Type
aws_eip.registry resource
aws_iam_access_key.registry resource
aws_iam_policy.registry resource
aws_iam_user.registry resource
aws_iam_user_policy_attachment.registry resource
aws_instance.registry resource
aws_kms_key.registry resource
aws_route53_record.registry resource
aws_route53_record.registry_private resource
aws_s3_bucket.registry resource
aws_s3_bucket_public_access_block.registry resource
aws_security_group.registry resource
aws_subnet.registry data source
aws_vpc.disco data source

Inputs

Name Description Type Default Required
ami_id The ID of the AMI that should be used for the registry. string n/a yes
availability_zone The availability zone into which the registry instance should be placed - should align with the subnet's zone. string n/a yes
cluster_domain The name of the domain under which your OpenShift cluster will reside. Will be used in the construction of DNS records for the registry in the public and private zones. string n/a yes
cluster_name The name you will be giving your OpenShift cluster in metadata.name in install-config.yaml. Will be used in the construction of DNS records for the registry in the public and private zones. string n/a yes
disk_gb The size of the disk, in GB, for the registry instance. Since the registry instance is expected to use S3 storage, can be small. number 20 no
flavor The instance type to use for the registry instance. string "t3.large" no
hostname The hostname to use when building the instance and creating Route 53 records for it. string "registry" no
instance_password The password to set for the ec2-user on the registry instance. string "" no
private_zone_id The Route53 Hosted Zone ID which contains the domain for creating private registry records. string n/a yes
private_zone_name The Route53 Hosted Zone name which contains the domain for creating private registry records. string n/a yes
public_zone The Route53 Hosted Zone ID which contains the domain for creating public registry records. string n/a yes
ssh_key_name The SSH public key to use for the proxy instance - must already exist as an aws_key_pair! string n/a yes
subnet_id The ID of the existing VPC subnet into which the instance should associate its default interface. string n/a yes

Outputs

Name Description
registry_instance Information about the registry instance.
s3_bucket The AWS S3 bucket and IAM credentials required to access it.

OCP Installer Submodule Documentation

Requirements

Name Version
terraform >= 1.4.2
aws 4.58.0

Providers

Name Version
aws 4.58.0

Modules

No modules.

Resources

Name Type
aws_iam_access_key.ocp_installer resource
aws_iam_policy.ocp_installer resource
aws_iam_user.ocp_installer resource
aws_iam_user_policy_attachment.ocp_installer resource

Inputs

Name Description Type Default Required
domain The full name of the domain which will be used to uniquely identify the IAM user created for OpenShift installation. string n/a yes

Outputs

Name Description
ocp_installer The IAM Access Key ID and Secret for the OpenShift installation user.