Unable to run jenkins/jenkins in Windows process isolation mode (revision 2004)

[anderson-dev]

Because my docker host (for development) is using Windows 10 Pro (rev 2004), in order to access named pipes, I must run docker in process isolation mode which necessitates my base image to be using the windowsservercore rev 2004 update. It seems that both adoptopenjdk and jenkins provide Dockerfiles/images only for ltsc versions but I'd like to make sure things work in development before I elevate to our server builds. So I attempted to build them myself. Everything works fine when running in Hyper-V isolation but I need to use process isolation in order to pass my docker named pipe through.

Also, when I use a simplified Dockerfile to only download the war and set as entrypoint, I'm able to get everything working fine. However, none of my settings are preserved in my mounted volume since the ENV JENKINS_HOME was never set. The moment I set JENKINS_HOME (even if setting all appropriate acls as done in the linked Dockerfile) and run in process isolation mode the errors indicated below occur. The first sign of error seems to be when trying to generate the master key for initial admin setup.

Version report

Jenkins and plugins versions report:

Dockerfile: https://github.com/jenkinsci/docker/blob/master/11/windows/windowsservercore-2019/hotspot/Dockerfile

• What Operating System are you using (both controller, and any agents involved in the problem)?

Windows 10 Pro (revision 2004)

Reproduction steps

• mkdir adoptopenjdk; cd adoptopenjdk
• Created new Dockerfile using the image layer info at hub.docker.com/adoptopenjdk as a template changing only the windowsservercore version to '2004' rather than 'lts2019' so that I can run in process isolation on my host
• docker build -t xenonet.local:5000/adoptopenjdk:11.0.10_9-jdk-hotspot-windowsservercore-2004 .
• cd ..
• mkdir jenkins; cd jenkins
• Created new Dockerfile using this Dockerfile as a template changing only the windowsservercore version to '2004' rather than 'lts2019' so that I can run in process isolation on my host
• docker build -t xenonet.local:5000/jenkins/jenkins-fix:jdk11-hotspot-windowsservercore-2004 .
• cd ..
• docker run --isolation process --rm -d -v jenkins-home:C:\ProgramData\jenkins\JenkinsHome -v \\.\pipe\docker_engine:\\.\pipe\docker_engine --name jenkins -p 8080:8080 xenonet.local:5000/jenkins/jenkins-fix:jdk11-hotspot-windowsservercore-2004

Results

Expected result:

1. no errors in docker logs jenkins
2. no errors when visiting localhost:8080 in web browser

Actual result:

error.log

[slide]

I think you need jenkins_home for the volume name.

[anderson-dev]

Thanks for your suggestion. I changed my volume name and still have the same issues. Regardless of volume name, hyper-v isolation works but process isolation doesn't

[slide]

I don't have a way of trying to replicate this, so I am not sure how to help. I know that the directory must NOT exist beforehand for Windows volumes. Does the C:\ProgramData\Jenkins\JenkinsHome directory exist before you run the container?

[anderson-dev]

It is created in the jenkins Dockerfile lines 15-35: https://github.com/jenkinsci/docker/blob/master/11/windows/windowsservercore-2019/hotspot/Dockerfile

ARG JENKINS_HOME=C:/ProgramData/Jenkins/JenkinsHome

ENV JENKINS_HOME $JENKINS_HOME
ENV JENKINS_AGENT_PORT ${agent_port}

# Jenkins home directory is a volume, so configuration and build history
# can be persisted and survive image upgrades
VOLUME $JENKINS_HOME

# Jenkins is run with user `jenkins`
# If you bind mount a volume from the host or a data container,
# ensure you use the same uid
RUN New-LocalUser -Name $env:user -AccountNeverExpires -Description 'Jenkins User' -NoPassword -UserMayNotChangePassword | Out-Null ; `
    Set-Localuser -Name $env:user -PasswordNeverExpires $true | Out-Null ; `
    Add-LocalGroupMember -Group "Administrators" -Member "${env:user}" ; `
    New-Item -Type Directory -Force -Path "C:/ProgramData/Jenkins" | Out-Null ; `
    icacls.exe "C:/ProgramData/Jenkins" /setowner ${env:user} | Out-Null ; `
    icacls.exe "C:/ProgramData/Jenkins" /inheritance:r | Out-Null ; `
    icacls.exe "C:/ProgramData/Jenkins" /grant:r $('{0}:(CI)(OI)(F)' -f $env:user) /grant 'Administrators:(CI)(OI)(F)' | Out-Null ; `
    icacls.exe "$env:JENKINS_HOME" /setowner ${env:user} | Out-Null ; `
    icacls.exe "$env:JENKINS_HOME" /grant:r $('{0}:(CI)(OI)(F)' -f $env:user) /grant 'Administrators:(CI)(OI)(F)' | Out-Null

[anderson-dev]

I'm creating a windows server 2019 ltsc vm to ensure it works for the kernel version currently published. I could create a windows 10 vm 2004 so that you may try and reproduce if you'd like.

[anderson-dev]

Unlike linux, where it always runs in process isolation mode allowing it to access a shared kernel, Windows runs in hyper-v isolation (VM) and is unable to share kernel resources such as named pipes with the host. A Windows container can be run in process isolation, however the container and the host OS must be using the same kernel version. I've verified that the official jenkins docker image works fine on Windows Server 2019 LTSC which currently is version 1809.

Windows 10, however, is currently on kernel version 21H2 according to https://docs.microsoft.com/en-us/windows/release-health/release-information. Microsoft official Windows docker base images are all based on Windows Server which has a different release schedule compared to Windows 10 https://docs.microsoft.com/en-us/windows/release-health/windows-server-release-info. Typically Windows Server is 2-3 kernel versions behind that of Windows 10. In order to run in process isolation using Windows 10 as a host, it requires carefully matching the kernel version with that of the docker container's base image. Because the jenkins official docker image only supports the kernel version 1809 which is currently the Long Term Support Channel version of Windows Server we must use a Windows 10 release that is very old. If the jenkins official docker image was rebuilt for a newer version of the windowsservercore base image, we could use it on Windows 10.

So the solution here is either to rebuild the official jenkins docker image using a newer version of the windowsservercore base image or simply give access to the Dockerfile and we could simply rebuild it ourselves.

[timja]

this may fix it: #1289