Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a job for creating on demand OpenShift clusters #2114

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Add a job for creating on demand OpenShift clusters #2114

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions cloud/jenkins/create-openshift-cluster.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
- job:
name: create-openshift-cluster
project-type: pipeline
description: |
Do not edit this job through the web!
pipeline-scm:
scm:
- git:
url: https://github.com/Percona-Lab/jenkins-pipelines.git
branches:
- 'master'
wipe-workspace: false
lightweight-checkout: true
script-path: cloud/jenkins/create_openshift_cluster.groovy
152 changes: 152 additions & 0 deletions cloud/jenkins/create_openshift_cluster.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
clusters=[]

void createCluster(String CLUSTER_SUFFIX){
clusters.add("${CLUSTER_SUFFIX}")

withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'openshift-cicd'], file(credentialsId: 'aws-openshift-41-key-pub', variable: 'AWS_NODES_KEY_PUB'), file(credentialsId: 'openshift4-secrets', variable: 'OPENSHIFT_CONF_FILE')]) {
sh """
platform_version=`echo "${params.PLATFORM_VER}" | awk -F. '{ printf("%d%03d%03d%03d\\n", \$1,\$2,\$3,\$4); }';`
version=`echo "4.12.0" | awk -F. '{ printf("%d%03d%03d%03d\\n", \$1,\$2,\$3,\$4); }';`
if [ \$platform_version -ge \$version ];then
POLICY="additionalTrustBundlePolicy: Proxyonly"
NETWORK_TYPE="OVNKubernetes"
else
POLICY=""
NETWORK_TYPE="OpenShiftSDN"
fi
mkdir -p openshift/${CLUSTER_SUFFIX}
cat <<-EOF > ./openshift/${CLUSTER_SUFFIX}/install-config.yaml
\$POLICY
apiVersion: v1
baseDomain: cd.percona.com
compute:
- architecture: amd64
hyperthreading: Enabled
name: worker
platform:
aws:
type: m5.2xlarge
replicas: 3
controlPlane:
architecture: amd64
hyperthreading: Enabled
name: master
platform: {}
replicas: 1
metadata:
creationTimestamp: null
name: openshift4-custom-jenkins-${CLUSTER_SUFFIX}-$BUILD_NUMBER
networking:
clusterNetwork:
- cidr: 10.128.0.0/14
hostPrefix: 23
machineNetwork:
- cidr: 10.0.0.0/16
networkType: \$NETWORK_TYPE
serviceNetwork:
- 172.30.0.0/16
platform:
aws:
region: eu-west-2
userTags:
iit-billing-tag: openshift
delete-cluster-after-hours: 8
team: cloud
product: pxc-operator

publish: External
EOF
cat $OPENSHIFT_CONF_FILE >> ./openshift/${CLUSTER_SUFFIX}/install-config.yaml
"""

sshagent(['aws-openshift-41-key']) {
sh """
/usr/local/bin/openshift-install create cluster --dir=./openshift/${CLUSTER_SUFFIX}
export KUBECONFIG=./openshift/${CLUSTER_SUFFIX}/auth/kubeconfig
"""
}
}
}

void shutdownCluster(String CLUSTER_SUFFIX) {
withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'openshift-cicd'], file(credentialsId: 'aws-openshift-41-key-pub', variable: 'AWS_NODES_KEY_PUB'), file(credentialsId: 'openshift-secret-file', variable: 'OPENSHIFT-CONF-FILE')]) {
sshagent(['aws-openshift-41-key']) {
sh """
source $HOME/google-cloud-sdk/path.bash.inc
export KUBECONFIG=$WORKSPACE/openshift/$CLUSTER_SUFFIX/auth/kubeconfig
for namespace in \$(kubectl get namespaces --no-headers | awk '{print \$1}' | grep -vE "^kube-|^openshift" | sed '/-operator/ s/^/1-/' | sort | sed 's/^1-//'); do
kubectl delete deployments --all -n \$namespace --force --grace-period=0 || true
kubectl delete sts --all -n \$namespace --force --grace-period=0 || true
kubectl delete replicasets --all -n \$namespace --force --grace-period=0 || true
kubectl delete poddisruptionbudget --all -n \$namespace --force --grace-period=0 || true
kubectl delete services --all -n \$namespace --force --grace-period=0 || true
kubectl delete pods --all -n \$namespace --force --grace-period=0 || true
done
kubectl get svc --all-namespaces || true

/usr/local/bin/openshift-install destroy cluster --dir=./openshift/$CLUSTER_SUFFIX || true
"""
}
}
}

pipeline {
parameters {
hors marked this conversation as resolved.
Show resolved Hide resolved
string(
defaultValue: '4.11.41',
description: 'OpenShift version to use',
name: 'PLATFORM_VER')
string(
defaultValue: '3',
description: 'Number of hours to keep the cluster alive',
name: 'CLUSTER_DURATION')
}
agent {
label 'docker'
}
options {
buildDiscarder(logRotator(daysToKeepStr: '-1', artifactDaysToKeepStr: '-1', numToKeepStr: '20', artifactNumToKeepStr: '20'))
skipDefaultCheckout()
}

stages {
stage('Prepare') {
steps {
sh '''
if [ ! -d $HOME/google-cloud-sdk/bin ]; then
rm -rf $HOME/google-cloud-sdk
curl https://sdk.cloud.google.com | bash
fi

source $HOME/google-cloud-sdk/path.bash.inc
gcloud components update kubectl
gcloud version

curl -s -L https://mirror.openshift.com/pub/openshift-v4/clients/ocp/$PLATFORM_VER/openshift-client-linux-$PLATFORM_VER.tar.gz \
| sudo tar -C /usr/local/bin --wildcards -zxvpf -
curl -s -L https://mirror.openshift.com/pub/openshift-v4/clients/ocp/$PLATFORM_VER/openshift-install-linux-$PLATFORM_VER.tar.gz \
| sudo tar -C /usr/local/bin --wildcards -zxvpf -
'''
}
}
stage('Create cluster') {
parallel {
stage('cluster1') {
steps {
createCluster('cluster1')
sleep(time:"${params.CLUSTER_DURATION}",unit:"HOURS")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the spot instance dies? Who will remove the cluster?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nobody, just like in our normal tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as a solution we can use jenkins master for running these jobs or specify non spot instances (pricier)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tplavcic The difference is you can run it for 12 hours and spot will have more time to die. I suggest to have https://cloud.cd.percona.com/job/pxc-operator-aws-remove-openshift officially

shutdownCluster('cluster1')
}
}
}
}
}
post {
always {
script {
clusters.each { shutdownCluster(it) }
}
deleteDir()
}
}
}