[ci.jenkins.io] Move controller (VM) to AWS

[dduportal]

This issue tracks the work to migrate ci.jenkins.io controller to the AWS Sponsored.

Current resources:

• User facing documentation: https://github.com/jenkins-infra/documentation/blob/main/ci.adoc
• Terraform definition of the current VM: https://github.com/jenkins-infra/azure/blob/main/ci.jenkins.io.tf
  • Utilizes a Terraform module: https://github.com/jenkins-infra/shared-tools/tree/main/terraform/modules/azure-jenkinsinfra-controller
  • With a dedicated subnet: https://github.com/jenkins-infra/azure-net/blob/5e16478469344f8eced5224e553a5f660794b9dd/vnets.tf#L204-L209 (there is a former subnet for the controller in the public_vnet for it but it is not used anymore)
  • 2 IPs (public for HTTP/HTTPS/JNLP and private for SSH) - https://github.com/jenkins-infra/azure/blob/e81b9697e4a9cedebf10e119b6a5a112e09b651f/ci.jenkins.io.tf#L57-L60
• VM is provisioned with puppet: https://github.com/jenkins-infra/jenkins-infra/blob/bb9e757b20dcec1da678948ef8b453d6d2cac928/manifests/site.pp#L77-L85

Target for the new VM:

• Same size (vCPUs and memory)
• Same disk topology (rootfs and data with jenkins home in it)
• For network: see [ci.jenkins.io] Define virtual networking for AWS #4320
  • We'll need a aws.ci.jenkins.io DNS A record (pointed by the CNAME ci.jenkins.io once migrated) to the public IPv4 of the controller, and a AAAA record with the public IPv6.
  • If need be, we'll define a private DNS A record aws.ci.jenkins.io so agents can reach the controller through private subnets
  • Inbound:
    • Anywhere HTTP + HTTPS on both IPv4 and IPv6
    • SSH from the (Azure private VPN) only. We'll set up the VPN routing of users to override routing like we did for the pkg.origin.jenkins.io, usage.jio and census.jio VMs.
    • Private subnets HTTP, HTTPS, JNLP
  • Outbound (IPv4 only):
    • HTTP+ HTTPS + HKP to everywhere
    • SSH to GitHub public IPs and to private subnets

[dduportal]

Discussed with @smerle33:

• [ci.jenkins.io] Define virtual networking for AWS #4320 reached a first milestone allowing starting work on the issue here
• Stephane prefers more the EKS topic than the VM topic: assigning VM to @dduportal
• Scheduling for current milestone

[dduportal]

Update:

• We need a DNS zone in AWS, delegated from our Azure DNS, to ensure we keep the DNS records up to date when changing IPs

• We want full in and out IPv6 support

• Controller must be in a public subnet, with both Network ACL and security groups restrictions.

• Big PR network + DNS + reports in feat: set up foundations for the VPC network terraform-aws-sponsorship#36

• Initial DNS delegation to validate it works in feat(DNS) delegate 'aws.ci.jenkins.io' subdomain to AWS route53 azure-net#317 (need an update with the new DNS zone name servers => automation incoming)

• Initial LDAP allow list update (need an update with the new DNS zone name servers => automation incoming) in feat(publick8s) add the new aws.ci.jio controller outbound IP to allowed list for LDAP kubernetes-management#5891

[dduportal]

Update:

• VM is now live (ref. feat(aws.ci.jenkins.io) create the first VM for controller terraform-aws-sponsorship#37)
• SSH access is good (required a few VPN routing changes, see attached issues above)
• VM packages upgraded, we can proceed to the new Puppet config (initial deployment) - feat(ci.jenkins.io) set up initial aws.ci.jenkins.io VM client data jenkins-infra#3730

[dduportal]

Update:

• Initial Puppet provisioning was successful. VM rebooted and access with nominative account is ok
  • Container jenkins is up and running
• Next steps:
  • Update LDAP allowed IPs to add the new CI IP
  • Update Network ACLs to allow inbound HTTP/HTTPS from internet
  • Update Puppet setup to support vhosts for both the internal hostname (aws.ci.jenkins.io) and the top-level CNAME (ci.jenkins.io)