HIVE-28339: Dockerfile and GitHub action for hive-ci-jenkins image

[zabetak]

What changes were proposed in this pull request?

1. Dockerfile for building the apache/hive-ci-jenkins image (to be used by http://ci.hive.apache.org/)
2. GitHub action for building and deploying the image to https://hub.docker.com/repository/docker/apache/hive-ci-jenkins/ on demand
3. Instructions/Suggestions for building, testing, and upgrading the Jenkins instance in CI.

Why are the changes needed?

1. Upgrade the Jenkins CI instance to the latest version
2. Host and maintain the Jenkins CI images under the apache/hive namespace
3. Provide guidelines/reference points for future updates

Does this PR introduce any user-facing change?

No

Is the change a dependency upgrade?

No

How was this patch tested?

The image produced from the Dockerfile here is already running production.

1. Followed roughly the steps described in the new README.md file for building and testing the image.
2. Pushed the image to https://hub.docker.com/repository/docker/apache/hive-ci-jenkins with lts-jdk21 tag.
3. Updated Kubernetes deployment for https://ci.hive.apache.org/ to use the apache/hive-ci-jenkins:lts-jdk21 image

[abstractdog]

I can see that Zoltan's repo contains an entrypoint script: https://github.com/kgyrtkirk/hive-test-kube/blob/master/htk-jenkins/entrypoint
isn't that missing from here? I guess we want to track it together with the Dockerfile

thanks for this effort so far, I totally agree with this initiative to include hive's ci infra code to the main repo

[zabetak]

@abstractdog The https://github.com/kgyrtkirk/hive-test-kube/blob/master/htk-jenkins/entrypoint does not appear necessary.

The JVM options are now passed using ENV JENKINS_JAVA_OPTS and the default configuration in jenkins_home is anyways overridden by the content of the persistent volume that is attached to the pod.

[kgyrtkirk]

these instructions work outside k8s; as it is I guess you can't launch a single job during the validation...
the right process should be:

• copy k8s volume
• create a new pod with the copied data
• use that to check things out

instead of launching with docker: I think you should document that (or remove these instructions)

[zabetak]

You can test various aspects but indeed launching jobs or connecting to the K8 nodes will not work.

The process outlined above indeed makes sense but wasn't very comfortable doing that in the actual cluster cause I was afraid of potential side-effects. If a second fully-working Jenkins instance comes up running it may interfere with the main one.

Anyways given that very few people have access to the K8 cluster it may be better to remove these instructions altogether for now.

[zabetak]

Removed upgrade instructions in 9870120

[kgyrtkirk]

yes - it was a surprise to me as well that k8s doesn't support VOLUME-s; the default config copy was a courtesy to make it usable right away if it gets deployed at a new location

I think putting this into the main repo is not the best - as you will need to deploy it and push the image to see what happens...so you will at most will be committing to the repo as an after-tought....keeping a separate un-confined repo will make it easier to track what and when was changed - but I might be alone with this view...

[abstractdog]

yes - it was a surprise to me as well that k8s doesn't support VOLUME-s; the default config copy was a courtesy to make it usable right away if it gets deployed at a new location

I think putting this into the main repo is not the best - as you will need to deploy it and push the image to see what happens...so you will at most will be committing to the repo as an after-tought....keeping a separate un-confined repo will make it easier to track what and when was changed - but I might be alone with this view...

I feel that keeping this image in the main repo is the way to go
how to manage the challenges that come with this approach is a different question, regarding which I have no idea now, honestly :)
this approach is not strictly "after-tought", only if it's proven to be impossible to test the Dockerfile changes in advance, I hope it's not the case, thinking about a github action to create and push an image with a custom tag from a PR like this (automatically or driven by a label)

[zabetak]

I agree with Zoltan that a separate repo would be beneficial for tracking CI and product changes separately. However, I am willing to sacrifice this separation for facilitating the discovery and contributions to the CI part.

At the same time, I feel that the testability aspect for Jenkins changes is independent of where the code is hosted. Even if this code is hosted in another repo we would still need a way to prove that changes in the Jenkins image do not break production. This is definitely an area with lots of room for improvements.

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[zabetak]

@abstractdog @kgyrtkirk Is there anything else apart from the repo debate holding off this PR? I want to get a feeling what is pending to finalize this work to see how to advance.

[okumin]

I confirmed docker build and docker run in README worked for me, and Jenkins was available on http://0.0.0.0:35000/.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the [email protected] list if the patch is in need of reviews.