Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support of windows container images (2019 and 2022) #819

Draft
wants to merge 14 commits into
base: main
Choose a base branch
from
+242 −245
Draft

feat: add support of windows container images (2019 and 2022) #819

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
f084365
wip: windows and docker
dduportal Aug 20, 2022
28326a9
wip: 2022 container provisioning is successful locally
dduportal Sep 24, 2023
4713bba
fixup: only try to build windows-2019 container for now
dduportal Sep 24, 2023
77cb920
fixup: simplify winrm user local as not needed in docker and format
dduportal Sep 24, 2023
8c3b188
wip use powershell to call packer
dduportal Sep 24, 2023
5facd97
debug: disable side tasks (faster feedback)
dduportal Sep 24, 2023
e136714
debug: only test windows-2019 containers (faster feedback)
dduportal Sep 24, 2023
3621721
wip
dduportal Sep 24, 2023
09cb60f
debug
dduportal Sep 24, 2023
e7573b7
fixup: powershell
dduportal Sep 24, 2023
596d40f
debug
dduportal Sep 25, 2023
e5301a4
debug with 2022
dduportal Sep 25, 2023
aa4ebbe
wip: remove docker windows exclusions
dduportal Sep 25, 2023
bca5d20
wip: test 2019 and 2020 in parallel
dduportal Sep 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
181 changes: 89 additions & 92 deletions Jenkinsfile_k8s
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,24 @@
// The node to be used depends on the build type: this function executes the pipeline code block provided as "body"
// into the correct node type based on the provided arguments
def withPackerNode(String packer_template, String compute_type, String cpu_architecture, Closure body) {
// Build ARM64 CPU Docker images on a native machine (faster than using the local qemu)
if (cpu_architecture == 'amd64' && compute_type == 'docker') {
node('linux-amd64-docker') {
// Docker images need to run with a local Docker Engine
if (compute_type == 'docker') {
String agentLabel = compute_type + '-' + packer_template
if (packer_template ==~ /ubuntu.*/) {
agentLabel = 'linux-amd64-docker'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review with @smerle33: missing arm64 case for Linux!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
agentLabel = 'linux-amd64-docker'
agentLabel = "linux-${cpu_architecture}-docker"

}
node(agentLabel) {
// New agent workspace specified as scripted requires an explicit checkout (compared to declarative)
checkout scm

// New agent means new packer project to initialize (plugins)
packerInitPlugins()
// packerInitPlugins()

// Authenticating to the GitHub API with an API token (auto-generated IAT, valid for 1 hour) provided to the environment variable PACKER_GITHUB_API_TOKEN
// to avoid hitting the rate limit. Ref. https://www.packer.io/docs/commands/init.
withCredentials([usernamePassword(credentialsId: 'github-app-infra',usernameVariable: 'UNUSED',passwordVariable: 'PACKER_GITHUB_API_TOKEN')]) {
powershell 'packer init .'
}

return body.call()
}
Expand All @@ -18,15 +28,24 @@ def withPackerNode(String packer_template, String compute_type, String cpu_archi
}
}

def universalShell(String command) {
if (isUnix()) {
// To allow using ASDF shims
withEnv(["PATH+ASDF=/home/jenkins/.asdf/shims:/home/jenkins/.asdf/bin"]) {
sh command
}
} else {
powershell command
}
}

// Initialize the packer project by installing the plugins in $PACKER_HOME_DIR/ - ref. https://www.packer.io/docs/configure
// This function must be called for each distinct agent but only one time (as plugins are OS and CPU specifics)
def packerInitPlugins() {
// Authenticating to the GitHub API with an API token (auto-generated IAT, valid for 1 hour) provided to the environment variable PACKER_GITHUB_API_TOKEN
// to avoid hitting the rate limit. Ref. https://www.packer.io/docs/commands/init.
withCredentials([usernamePassword(credentialsId: 'github-app-infra',usernameVariable: 'UNUSED',passwordVariable: 'PACKER_GITHUB_API_TOKEN')]) {
// Cleanup any remnant of packer plugins on this agent
sh 'rm -rf /home/jenkins/.config /home/jenkins/.packer*'
sh 'packer init ./'
universalShell 'packer init .'
}
}

Expand Down Expand Up @@ -56,80 +75,79 @@ pipeline {
options {
timeout(time: 120, unit: 'MINUTES')
}
environment {
// To allow using ASDF shims
PATH = "${env.PATH}:/home/jenkins/.asdf/shims:/home/jenkins/.asdf/bin"
}
stages {
stage('Side Tasks') {
environment {
DRYRUN = "${env.BRANCH_IS_PRIMARY ? 'false' : 'true'}"
}
parallel {
stage('Packer Init') {
steps {
// Call the initializing function once for the default agent
script {
packerInitPlugins()
}
}
}
stage('GC on Azure') {
environment {
PACKER_AZURE = credentials('packer-azure-serviceprincipal')
}
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh 'az login --service-principal -u "$PACKER_AZURE_CLIENT_ID" -p "$PACKER_AZURE_CLIENT_SECRET" -t "$PACKER_AZURE_TENANT_ID"'
sh 'az account set -s "$PACKER_AZURE_SUBSCRIPTION_ID"'
sh './cleanup/azure_gallery_images.sh 1 dev'
sh './cleanup/azure_gallery_images.sh 7 staging'
sh './cleanup/azure.sh 1 dev'
sh './cleanup/azure.sh 1 staging'
sh './cleanup/azure.sh 1 prod'
}
}
}
stage('Updatecli') {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
script {
// TODO: Implement https://github.com/jenkins-infra/pipeline-library/issues/518 to allow using the updatecli() library function
withCredentials([
usernamePassword(
credentialsId: 'github-app-updatecli-on-jenkins-infra',
usernameVariable: 'USERNAME_VALUE', // Setting this variable is mandatory, even if of not used when the credentials is a githubApp one
passwordVariable: 'UPDATECLI_GITHUB_TOKEN'
)
]) {
sh 'updatecli version'
sh 'updatecli diff --values ./updatecli/values.yaml --config ./updatecli/updatecli.d'
if (env.BRANCH_IS_PRIMARY) {
sh 'updatecli apply --values ./updatecli/values.yaml --config ./updatecli/updatecli.d'
}
}
}
}
}
}
}
}
// stage('Side Tasks') {
// environment {
// DRYRUN = "${env.BRANCH_IS_PRIMARY ? 'false' : 'true'}"
// }
// parallel {
// stage('Packer Init') {
// steps {
// // Call the initializing function once for the default agent
// script {
// packerInitPlugins()
// }
// }
// }
// stage('GC on Azure') {
// environment {
// PACKER_AZURE = credentials('packer-azure-serviceprincipal')
// }
// steps {
// catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
// sh 'az login --service-principal -u "$PACKER_AZURE_CLIENT_ID" -p "$PACKER_AZURE_CLIENT_SECRET" -t "$PACKER_AZURE_TENANT_ID"'
// sh 'az account set -s "$PACKER_AZURE_SUBSCRIPTION_ID"'
// sh './cleanup/azure_gallery_images.sh 1 dev'
// sh './cleanup/azure_gallery_images.sh 7 staging'
// sh './cleanup/azure.sh 1 dev'
// sh './cleanup/azure.sh 1 staging'
// sh './cleanup/azure.sh 1 prod'
// }
// }
// }
// stage('Updatecli') {
// steps {
// catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
// script {
// // TODO: Implement https://github.com/jenkins-infra/pipeline-library/issues/518 to allow using the updatecli() library function
// withCredentials([
// usernamePassword(
// credentialsId: 'github-app-updatecli-on-jenkins-infra',
// usernameVariable: 'USERNAME_VALUE', // Setting this variable is mandatory, even if of not used when the credentials is a githubApp one
// passwordVariable: 'UPDATECLI_GITHUB_TOKEN'
// )
// ]) {
// sh 'updatecli version'
// sh 'updatecli diff --values ./updatecli/values.yaml --config ./updatecli/updatecli.d'
// if (env.BRANCH_IS_PRIMARY) {
// sh 'updatecli apply --values ./updatecli/values.yaml --config ./updatecli/updatecli.d'
// }
// }
// }
// }
// }
// }
// }
// }
stage('Packer Images') {
matrix {
axes {
axis {
name 'cpu_architecture'
values 'amd64', 'arm64'
// values 'amd64', 'arm64'
values 'amd64'
}
axis {
name 'agent_type'
// make sure to port any addition to the list of agent types to the Build Docker Manifest stage if it's docker related
values 'ubuntu-22.04', 'windows-2019', 'windows-2022'
// values 'ubuntu-22.04', 'windows-2019', 'windows-2022'
values 'windows-2019', 'windows-2022'
}
axis {
name 'compute_type'
// "azure-arm" stands for "Azure Resource Manager", unrelated to arm64 CPU
values 'azure-arm', 'docker'
// values 'azure-arm', 'docker'
values 'docker'
}
}
excludes {
Expand All @@ -148,27 +166,6 @@ pipeline {
values 'azure-arm'
}
}
// No build on Windows or Docker, not yet implemented
exclude {
axis {
name 'agent_type'
values 'windows-2019'
}
axis {
name 'compute_type'
values 'docker'
}
}
exclude {
axis {
name 'agent_type'
values 'windows-2022'
}
axis {
name 'compute_type'
values 'docker'
}
}
}
environment {
// Defines the following environment variables: AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID, AZURE_SUBSCRIPTION_ID
Expand Down Expand Up @@ -203,14 +200,14 @@ pipeline {

withPackerNode(pkr_var_agent_os_type + '-' + pkr_var_agent_os_version , pkr_var_image_type, pkr_var_architecture) {
// Validate template (for all elements)
sh 'PACKER_LOG=1 packer validate ./'
powershell 'packer validate .'

// Execute build only for this matrix cell's setup
retry(count: 2, conditions: [kubernetesAgent(handleNonKubernetes: true), nonresumable()]) {
sh 'packer build -timestamp-ui -force -only="${PKR_VAR_image_type}.${PKR_VAR_agent_os_type}" ./'
powershell 'packer build -timestamp-ui -force -only="${PKR_VAR_image_type}.${PKR_VAR_agent_os_type}" .'
// adding manually a cpu architecture tag to the docker image
if (pkr_var_image_type == 'docker') {
sh 'docker tag "jenkinsciinfra/jenkins-agent-${PKR_VAR_agent_os_type}-${PKR_VAR_agent_os_version}:latest" "jenkinsciinfra/jenkins-agent-${PKR_VAR_agent_os_type}-${PKR_VAR_agent_os_version}:${PKR_VAR_architecture}"'
powershell 'docker tag "jenkinsciinfra/jenkins-agent-${PKR_VAR_agent_os_type}-${PKR_VAR_agent_os_version}:latest" "jenkinsciinfra/jenkins-agent-${PKR_VAR_agent_os_type}-${PKR_VAR_agent_os_version}:${PKR_VAR_architecture}"'
}
}
// if docker and building a tag, push to dockerhub from inside the node
Expand All @@ -219,7 +216,7 @@ pipeline {
stage('Publish all tags for Docker image') {
echo "Pushing jenkinsciinfra/jenkins-agent-${pkr_var_agent_os_type}:${pkr_var_tag_name} & jenkinsciinfra/jenkins-agent-${pkr_var_agent_os_type}:latest for ${pkr_var_architecture}"
infra.withDockerPushCredentials {
sh 'docker push --all-tags jenkinsciinfra/jenkins-agent-${agent_type}'
powershell 'docker push --all-tags jenkinsciinfra/jenkins-agent-${agent_type}'
}
}
}
Expand Down
22 changes: 20 additions & 2 deletions build-jenkins-agent-ubuntu.pkr.hcl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,24 @@
build {
source "docker.base" {
name = "ubuntu"

# Use official ubuntu image, with the tag set to the numeric version
image = "${var.agent_os_type}:${var.agent_os_version}"

# To improve audit and garbage collecting, we provide "labels" to the image
changes = [
"LABEL timestamp = ${local.now_unix_timestamp}",
"LABEL version = ${var.image_version}",
"LABEL scm_ref = ${var.scm_ref}",
"LABEL build_type = ${var.build_type}",
"ENV LANG=${var.locale}",
"ENV LANGUAGE=${element(split(".", var.locale), 0)}:${element(split("_", var.locale), 0)}",
"ENV LC_ALL=${var.locale}",
"ENV AGENT_WORKDIR=/home/jenkins/agent",
"WORKDIR /home/jenkins",
"ENTRYPOINT [\"/usr/local/bin/jenkins-agent\"]",
"USER jenkins",
]
}

source "azure-arm.base" {
Expand Down Expand Up @@ -54,9 +72,9 @@ build {
}

post-processor "docker-tag" {
only = ["docker.ubuntu"]
only = ["docker.ubuntu"]
# TODO specify architecture in image name with local.image_name
repository = format("%s/jenkins-agent-%s-%s",var.docker_namespace, var.agent_os_type, var.agent_os_version)
repository = format("%s/jenkins-agent-%s-%s", var.docker_namespace, var.agent_os_type, var.agent_os_version)
tags = [var.image_version, "latest"]
}
}
Loading