Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Infrastructure #75

Closed
wants to merge 42 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
997580b
adds the trust policy, the role, a readme to include some comments ab…
carlosvalarezo Aug 17, 2023
706fbd6
removes github directory from the gitignore file in order to use gith…
carlosvalarezo Aug 17, 2023
4869c36
creates the cv-eks-cluster-role role using githubactions to avoid doi…
carlosvalarezo Aug 17, 2023
1c19650
removes action not found
carlosvalarezo Aug 17, 2023
73a5482
removes action not found
carlosvalarezo Aug 17, 2023
8ddc0d1
adds aws-default-region since This is a problem with awscli trying to…
carlosvalarezo Aug 17, 2023
96b2c13
adds aws configure command rather than aws iam create-role directly
carlosvalarezo Aug 17, 2023
f7a4308
uses env to get the secrets and use them in aws configure
carlosvalarezo Aug 18, 2023
aae39af
uses echo to check values
carlosvalarezo Aug 18, 2023
bbc0f8c
uses DEVELOP as part of the env variable in order to retrieve the sec…
carlosvalarezo Aug 18, 2023
20ef118
removes unnecesary echo
carlosvalarezo Aug 18, 2023
38887d5
uses env to validate the secrets available
carlosvalarezo Aug 18, 2023
5cdc4b3
uses env to validate the secrets available
carlosvalarezo Aug 18, 2023
8088326
adds enviroment o define the environment
carlosvalarezo Aug 18, 2023
faed0ad
uses env variables to pass secrets to ther aws configure command
carlosvalarezo Aug 18, 2023
b0f29b4
checks if the credentials file was created
carlosvalarezo Aug 18, 2023
9e1e8b5
checks if the credentials file was created
carlosvalarezo Aug 18, 2023
18202a8
checks if the credentials file was created
carlosvalarezo Aug 18, 2023
a09fa5b
adds region to the command and validates the creds to execute commands
carlosvalarezo Aug 18, 2023
86cc2a0
adds region to the command and validates the creds to execute commands
carlosvalarezo Aug 18, 2023
22535ee
adds region to the command and validates the creds to execute commands
carlosvalarezo Aug 18, 2023
ffab863
adds region to the command and validates the creds to execute commands
carlosvalarezo Aug 18, 2023
f9a9d0f
adds region to the command and validates the creds to execute commands
carlosvalarezo Aug 18, 2023
b50e1ec
sets the env vars for all the steps
carlosvalarezo Aug 18, 2023
d3f9e9d
updates the name of the job and adds the step to attach the policy to…
carlosvalarezo Aug 21, 2023
cc419cd
updates the ubuntu runner and installs jq to extract values later
carlosvalarezo Aug 21, 2023
8b00ebb
moves to env vars cluster role and policy name and use them in the re…
carlosvalarezo Aug 21, 2023
06408da
removes unnecesary install of jq and updates the name of the env vars
carlosvalarezo Aug 21, 2023
2ea1787
adds the steps to create the role and to attach the policy for the no…
carlosvalarezo Aug 21, 2023
5faa55f
configures the yaml file to create the k8s eks cluster via eksctl
carlosvalarezo Aug 21, 2023
3a0feba
setup the runner installing eksctl
carlosvalarezo Aug 21, 2023
ad3f00f
setup the runner installing eksctl
carlosvalarezo Aug 21, 2023
2fe543c
setup the runner installing eksctl
carlosvalarezo Aug 21, 2023
98946d6
updates the path of the script file
carlosvalarezo Aug 21, 2023
f899779
updates the path of the script file
carlosvalarezo Aug 21, 2023
9ea53fc
updates the path of the script file
carlosvalarezo Aug 21, 2023
c30549e
creates the cluster
carlosvalarezo Aug 21, 2023
45414eb
uses the correct file to create the cluster using eksctl
carlosvalarezo Aug 21, 2023
ac8f9fe
create the ssh key files getting the values from the secrets repo
carlosvalarezo Aug 21, 2023
72490f4
creates the .ssh directory
carlosvalarezo Aug 21, 2023
d51df7f
adds new comments and separates the setup of the roles and policies f…
carlosvalarezo Aug 21, 2023
e7f5dff
moves to infrastructure branch and updates the workflow
carlosvalarezo Aug 21, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/scripts/eksctl-install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/bin/bash

ARCH=amd64
PLATFORM=$(uname -s)_$ARCH

curl -sLO "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_$PLATFORM.tar.gz"

# (Optional) Verify checksum
curl -sL "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_checksums.txt" | grep $PLATFORM | sha256sum --check

tar -xzf eksctl_$PLATFORM.tar.gz -C /tmp && rm eksctl_$PLATFORM.tar.gz

sudo mv /tmp/eksctl /usr/local/bin
65 changes: 65 additions & 0 deletions .github/workflows/cv-infrastructure.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: Create AWS infrastructure
on:
push:
branches:
- infrastructure
jobs:
setup_roles_and_policies:
runs-on: ubuntu-latest
environment:
name: develop
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
CLUSTER_ROLE_NAME: cv-eks-cluster-role
CLUSTER_POLICY_NAME: cv-eks-cluster-policy
NODEGROUP_ROLE_NAME: cv-eks-nodegroup-role
NODEGROUP_POLICY_NAME: cv-eks-nodegroup-policy
steps:
- name: Checkout code
uses: actions/[email protected]

- name: Setup the runner
run: sh $GITHUB_WORKSPACE/.github/scripts/eksctl-install.sh

- name: Setup AWS credentials
run: |
aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID}
aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY}
aws configure set default.region us-west-1

- name: Creates the role that will assume the trust policy to deal with the cluster
run: |
aws iam create-role --role-name ${CLUSTER_ROLE_NAME} --assume-role-policy-document file://${GITHUB_WORKSPACE}/eksk8s/eks-cluster-trust-policy.json

- name: Attaches the policy to the new role
run: |
aws iam put-role-policy --role-name ${CLUSTER_ROLE_NAME} --policy-name ${CLUSTER_POLICY_NAME} --policy-document file://${GITHUB_WORKSPACE}/eksk8s/eks-cluster-policy.json

- name: Creates the role to manage the node-group
run: |
aws iam create-role --role-name ${NODEGROUP_ROLE_NAME} --assume-role-policy-document file://${GITHUB_WORKSPACE}/eksk8s/eks-nodegroup-trust-policy.json

- name: Attaches the policy for the node-group
run: |
aws iam put-role-policy --role-name ${NODEGROUP_ROLE_NAME} --policy-name ${NODEGROUP_POLICY_NAME} --policy-document file://${GITHUB_WORKSPACE}/eksk8s/eks-nodegroup-policy.json

create_the_cluster:
runs-on: ubuntu-latest
needs: setup_roles_and_policies
environment:
name: develop
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
SSH_PUBLIC_KEY: ${{ secrets.SSH_PUBLIC_KEY }}
steps:
- name: Setup the keys used to create the k8s cluster
run: |
mkdir $HOME/.ssh/
echo $SSH_PRIVATE_KEY > $HOME/.ssh/k8s
echo $SSH_PUBLIC_KEY > $HOME/.ssh/k8s.pub
- name: Creates the k8s cluster using eksctl
run: |
eksctl create cluster -f ${GITHUB_WORKSPACE}/eksk8s/cluster-config.yaml
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,3 @@ htmlcov/
.envrc
.direnv

.github/**
24 changes: 24 additions & 0 deletions eksk8s/README.MD
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
With regard to the file eks-cluster-policy.json:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"eks:CreateCluster",
"eks:DescribeCluster",
"eks:UpdateClusterConfig",
"eks:UpdateClusterVersion",
"ssm:GetParameters"
],
"Resource": "*" // * means that the already defined actions could be applied to any EKS cluster in the account
}
]
}

In the file eksk8s/cluster-config.yaml the key nodeGroups.iam.instanceRoleARN has the value=arn:aws:iam::62132xxxx821:role/cv-eks-nodegroup-role. In a real project this file should be created using jinja/cookicutter in a previous step. This file should never have hard-coded values. However, for educational purposes/save time these values are hardcoded in the file.

This key pair was created in aws console. It is possible to do it using the command to create it. This step involves pushing (uploading) the keys to aws via aws cli or somehow.

The subnets were also created by hand in the aws console.
31 changes: 31 additions & 0 deletions eksk8s/cluster-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
name: cv-eks-cluster
region: us-west-1

vpc:
id: vpc-04e8935eb17607398
subnets:
public:
us-west-1a:
id: subnet-08375f54b10ac81a2
us-west-1b:
id: subnet-0a8eea0bb2b8bd056
clusterEndpoints:
publicAccess: true
securityGroup: sg-074fc7db95c8a8141

nodeGroups:
- name: cv-eks-nodegroup
iam:
instanceRoleARN: arn:aws:iam::621328949821:role/cv-eks-nodegroup-role
instanceType: t2.micro
desiredCapacity: 2
ssh:
allow: true
publicKeyPath: ~/.ssh/k8s.pub
subnets:
- subnet-08375f54b10ac81a2
- subnet-0a8eea0bb2b8bd056
17 changes: 17 additions & 0 deletions eksk8s/eks-cluster-policy.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"eks:CreateCluster",
"eks:DescribeCluster",
"eks:UpdateClusterConfig",
"eks:UpdateClusterVersion",
"ssm:GetParameters"
],
"Resource": "*"
}
]
}

13 changes: 13 additions & 0 deletions eksk8s/eks-cluster-trust-policy.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "eks.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}

70 changes: 70 additions & 0 deletions eksk8s/eks-nodegroup-policy.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:UnassignPrivateIpAddresses",
"ec2:AssignPrivateIpAddresses",
"ecr:GetAuthorizationToken",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:DescribeInstances",
"ec2:AssociateRouteTable",
"ec2:DeleteVolume",
"logs:CreateLogStream",
"eks:DescribeNodegroup",
"autoscaling:DescribeAutoScalingGroups",
"ec2:CreateRoute",
"ec2:DescribeVolumes",
"ec2:DescribeRouteTables",
"ecr:BatchCheckLayerAvailability",
"ec2:DetachVolume",
"ec2:ModifyVolume",
"eks:ListNodegroups",
"ec2:CreateTags",
"autoscaling:DescribeTags",
"ecr:GetDownloadUrlForLayer",
"ec2:CreateRouteTable",
"ec2:RunInstances",
"ec2:DisassociateRouteTable",
"logs:CreateLogGroup",
"ec2:CreateVolume",
"ecr:PutImage",
"eks:DeleteNodegroup",
"ecr:BatchGetImage",
"eks:UpdateNodegroupConfig",
"ec2:DescribeSubnets",
"ecr:InitiateLayerUpload",
"ec2:AttachVolume",
"ec2:DeleteTags",
"logs:DescribeLogStreams",
"autoscaling:DescribeLaunchConfigurations",
"ecr:UploadLayerPart",
"ecr:ListImages",
"iam:PassRole",
"ec2:CreateNetworkInterface",
"ec2:DescribeNetworkInterfaces",
"ec2:DeleteNetworkInterface",
"ec2:CreateSecurityGroup",
"ec2:DetachNetworkInterface",
"ec2:ModifyNetworkInterfaceAttribute",
"ec2:AttachNetworkInterface",
"ecr:CompleteLayerUpload",
"ecr:DescribeRepositories",
"ec2:ModifyInstanceAttribute",
"ec2:RebootInstances",
"ec2:TerminateInstances",
"logs:DescribeLogGroups",
"ec2:DeleteRoute",
"eks:CreateNodegroup",
"logs:PutLogEvents",
"ec2:DescribeSecurityGroups",
"ec2:DeleteSecurityGroup",
"ecr:GetRepositoryPolicy"

],
"Resource": "*"
}
]
}

20 changes: 20 additions & 0 deletions eksk8s/eks-nodegroup-trust-policy.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
},
{
"Effect": "Allow",
"Principal": {
"Service": "eks.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}