From 6456f616128df8836a1b64858424a795e28bdc5c Mon Sep 17 00:00:00 2001 From: Christoph Szymanski Date: Wed, 8 May 2024 12:51:22 +0200 Subject: [PATCH] Rem cx-server references from infrastructure documentation (#4886) * Rem cx-server references from infrastructure documentation * Rem reference to cx server in plugin doc * Rem cx-server reference from abap doc * Rem cx-server reference from gcts cod * Rem cx-server ref from guided tour * Rem unused links * Rem unused refs * remove cx-server from development.md --------- Co-authored-by: Oliver Feldmann Co-authored-by: tiloKo <70266685+tiloKo@users.noreply.github.com> --- DEVELOPMENT.md | 2 +- documentation/docs/guidedtour.md | 2 +- .../docs/infrastructure/customjenkins.md | 14 -- documentation/docs/infrastructure/overview.md | 123 +----------------- .../docs/scenarios/abapEnvironmentAddons.md | 2 +- documentation/docs/scenarios/gCTS_Scenario.md | 2 +- 6 files changed, 5 insertions(+), 140 deletions(-) diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 7e3d8a003a..8ef0ed8a79 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -70,7 +70,7 @@ you need to do the following in addition: * [Install Groovy](https://groovy-lang.org/install.html) * [Install Maven](https://maven.apache.org/install.html) -* Get a local Jenkins installed: Use for example [cx-server](https://github.com/SAP/devops-docker-cx-server) +* Get a local Jenkins installed ### Jenkins pipelines diff --git a/documentation/docs/guidedtour.md b/documentation/docs/guidedtour.md index ff8c53573f..71fdbf472b 100644 --- a/documentation/docs/guidedtour.md +++ b/documentation/docs/guidedtour.md @@ -50,7 +50,7 @@ Copy the sources of the application into your own Git repository. While we will 1. Save your changes to your remote repository. -1. To set up a Jenkins job for your repository, open the Jenkins UI under `http://:` and choose **New Item**. Per default, the `cx-server` starts Jenkins on HTTP port `80`. For more information, see the [Jenkins User Documentation][jenkins-io-documentation]. +1. To set up a Jenkins job for your repository, open the Jenkins UI under `http://:` and choose **New Item**. For more information, see the [Jenkins User Documentation][jenkins-io-documentation]. ![Clicke New Item](images/JenkinsHomeMenu-1.png "Jenkins Home Menu") diff --git a/documentation/docs/infrastructure/customjenkins.md b/documentation/docs/infrastructure/customjenkins.md index 29d33db9a9..54e8ea0c40 100644 --- a/documentation/docs/infrastructure/customjenkins.md +++ b/documentation/docs/infrastructure/customjenkins.md @@ -24,18 +24,6 @@ docker run ... -v /var/run/docker.sock:/var/run/docker.sock ... Project "Piper" requires a set of plugins installed on your Jenkins server. This set may evolve in the future. Make sure that all plugins of the appropriate versions are installed. -The Cx server repository contains an [up-to-date list][devops-cxs-plugins] of the required plugins. To ease the installation, download the list with the following command and use the [Jenkins client][jenkins-doc-client]: - -``` -curl -o plugins.txt https://raw.githubusercontent.com/SAP/devops-docker-cx-server/master/jenkins-master/plugins.txt -``` - -On the Jenkins server, run the following command with a user that has administration rights: - -``` -cat plugins.txt | awk '{system("java " "-jar jenkins-cli.jar -s http://localhost:8080 -auth ${ADM_USER}:${ADM_PASSWD} install-plugin " $1)}' -``` - ## Shared Library Shared libraries extending the Jenkins pipeline are defined within the Jenkins system configuration. A library is defined by a link to its source repository and an appropriate version identifier. To add the project "Piper"s library, execute the following steps: @@ -100,7 +88,5 @@ If you face such a [user permission issue][piper-issue-781], choose between the [docker-install]: https://docs.docker.com/install [dockerhub-node]: https://hub.docker.com/_/node/ [docker-getstarted]: https://docs.docker.com/get-started/ -[jenkins-doc-client]: https://jenkins.io/doc/book/managing/cli/ [jenkins-docker-image]: https://github.com/jenkinsci/docker/ [piper-issue-781]: https://github.com/SAP/jenkins-library/issues/781 -[devops-cxs-plugins]: https://github.com/SAP/devops-docker-cx-server/blob/master/jenkins-master/plugins.txt diff --git a/documentation/docs/infrastructure/overview.md b/documentation/docs/infrastructure/overview.md index aaf4451044..8fae679b6b 100644 --- a/documentation/docs/infrastructure/overview.md +++ b/documentation/docs/infrastructure/overview.md @@ -1,128 +1,7 @@ # Infrastructure -Besides SAP specific Jenkins library steps and out-of-the-box pipelines, project "Piper" offers also documentation and tooling to start the corresponding Jenkins server with all the configuration required to run project "Piper" pipelines. - -The core of the Jenkins infrastructure tooling is a set of [Docker images][docker-images]. -There is a main Docker image containing a preconfigured Jenkins and several tooling images used in the specific project "Piper" steps. - -!!! info "Docker Hub rate limiting" - Please be aware that Docker Hub as rate limiting active which might cause project "Piper" pipelines to fail. - Refer to the [page dedicated to docker rate limiting for details and solutions][resources-docker-rate-limit]. - -The document and the linked resources explain the various ways of starting such a Jenkins server based on these Docker images. - -## Cx Server (Recommended) - -Cx Server is a life-cycle management tool to bootstrap a pre-configured Jenkins instance within minutes on your own (virtual) server. -It uses the Docker images mentioned above. -As it would be cumbersome to start the Docker image manually with all required parameters and sidecar images, this command line tool automates the bootstraping. - -### Setting up a Jenkins master - -For the following steps you will need a server or another machine which has Docker installed and configured. - -To get started, initialize the Cx Server by using this `docker run` command: - -```sh -docker run -it --rm -u $(id -u):$(id -g) -v "${PWD}":/cx-server/mount/ ppiper/cx-server-companion:latest init-cx-server -``` - -This creates a few files in your current working directory. -The shell script `cx-server` and the configuration file `server.cfg` are of special interest. - -Now, you can start the Jenkins server by using the following command: - -```sh -chmod +x ./cx-server -./cx-server start -``` - -For more information on the Cx Server and how to customize your Jenkins, have a look at the [Operations Guide for Cx Server][devops-docker-images-cxs-guide]. - -### Setting up Jenkins agents - -With more and more qualities checked automatically in the pipeline, more and more resources are required to handle the workload. -This section shows how to scale the pipeline by adding [Jenkins build agents][build-agents]. - -However, before setting up agents please consider also other ways to scale the build infrastructure. -It might be an option to have only one Jenkins master with lots of resources (cpu cores, memory) per project or team. -This has the advantage of bringing more configuration flexibility and isolation for the individual teams but has the disadvantage that parts of the configuration have to be maintained twice. -Furthermore, having agents and thus network communication between the build servers increases the risk of failures. - -To add an agent to the Jenkins master, please make sure to fulfil the following requirements similar to the ones for the Jenkins master: - -- Access to a new server which runs on Linux -- Docker installed on this server - -The connection between the master and the agents will be established via ssh. -As the Jenkins master runs in a Docker container, the ssh setup steps, such as creating and storing a private/public key pair or maintaining the konwn hosts file has to be done inside this container. - -To execute these steps inside the container, execute the following command on the server where the Jenkins master is running: - -```bash -docker exec -it cx-jenkins-master bash -``` - -Inside the container make sure to be able to access the server where the Jenkins agent should be started by running the following command. As user you should use a user which is able to execute `docker` commands, i.e. starting a docker container. - -```bash -ssh @ -``` - -To be able to access the agent via ssh with the command above you might need to generate a new ssh key with `ssh-keygen`, store it in the `.ssh` folder and register the public key on the agent server. -You might also need to add server’s fingerprint to the list of known hosts. -For more information around establishing a ssh connection please consult the [ssh documentation][ssh-documentation]. - -To setup a new Jenkins agent, open "Manage Jenkins" > "Manage Nodes" > "New Nodes" and create a new "Permanent Agent" - -Please define `/var/jenkins_home` as "Remote root directory". -The launch method has to be "Launch agent via execution of command on the master" and the command should be: -`./var/jenkins_home/launch-jenkins-agent.sh [image]`. -User and host should equal the values you used above to test the ssh connection. - -The following picture shows an example configuration. - -![Agent Setup](../images/agent.png "Agent Setup") - -## Kubernetes (Experimental) - -Hosting Jenkins master and agents means that we bind the required resources to the purpose of executing builds. -There are good chances that, these resources stay idle for the most part of the day, i.e. if you have high peak loads. -Autoscaling of the infrastructure solves such a problem. -Instead of reserving the resources proactively, the pipeline creates the Jenkins agents dynamically on a Kubernetes cluster during the execution. -Once the agent completes the dedicated task, it is deleted and the resources are freed. -Project "Piper" supports running the pipeline as well as individual steps in a Kubernetes Cluster. -Please note that this feature is currently only experimental. - -To setup the Jenkins master in Kubernetes you can use helm. -The documentation to install Jenkins using helm can be found [here][jenkins-helm]. - -To use the Jenkins image provided by project Piper, pass `ppiper/jenkins-master` as a value for the `Master.Image` command line argument while deploying Jenkins to Kubernetes. - -The successfully completed deployment consists of a Jenkins pod with port 80 and 50000 exposed for HTTP and internal JNLP traffic respectively. -The deployment also creates two services each to listen to incoming HTTP traffic on port 80 and the internal JNLP traffic on port 50000. -Please note that in this example setup, the SSL/TLS termination happens at the load balancer, hence all the traffic between a load balancer and the Jenkins pod is unencrypted. - -Project "Piper" needs an environment variable set in the Jenkins to run the workload in Kubernetes. -In order to set the environment variable, navigate to "Manage Jenkins" > "Configure System" > "Global Properties". -Add an environment variable ON_K8S and set the value to true: - -![Environment Variable ON_K8S](../images/env.png "Environment Variable ON_K8S") - -Afterwards, you should be able to run project "Piper" pipelines in Kubernetes. - ## Custom Jenkins -### On your own: Custom Jenkins Setup - -If you use your own Jenkins installation, you need to care for the configuration that is specific to project "Piper". -This option should only be considered if you know why you need it, otherwise using the Cx Server life-cycle management makes your life much easier. -If you choose to go this path, follow the [Custom Jenkins Setup guide][resources-custom-jenkins]. +To run project "Piper", you will need your own Jenkins installation, and you need to care for the configuration that is specific to project "Piper". Please follow the [Custom Jenkins Setup guide][resources-custom-jenkins]. -[devops-docker-images-cxs-guide]: https://github.com/SAP/devops-docker-cx-server/blob/master/docs/operations/cx-server-operations-guide.md -[docker-images]: https://hub.docker.com/u/ppiper -[resources-docker-rate-limit]: docker-rate-limit.md [resources-custom-jenkins]: customjenkins.md -[build-agents]: https://wiki.jenkins.io/display/jenkins/distributed+builds -[ssh-documentation]: https://www.openssh.com/manual.html -[jenkins-helm]: https://github.com/helm/charts/tree/master/stable/jenkins diff --git a/documentation/docs/scenarios/abapEnvironmentAddons.md b/documentation/docs/scenarios/abapEnvironmentAddons.md index 1baf903bd5..eddfed2137 100644 --- a/documentation/docs/scenarios/abapEnvironmentAddons.md +++ b/documentation/docs/scenarios/abapEnvironmentAddons.md @@ -107,7 +107,7 @@ There are several prerequisites to run the pipeline for building an ABAP Enviro #### Jenkins Server -The pipeline responsible for building ABAP add-ons has been created specifically for [Jenkins](https://www.jenkins.io). Therefore, a Jenkins Server is required. The [piper project](https://sap.github.io/jenkins-library/guidedtour/) provides with [Cx Server](https://www.project-piper.io/infrastructure/overview/#cx-server-recommended) a life-cycle management tool to bootstrap a pre-configured Jenkins instance, which already includes the necessary configuration. Of course, it is also possible to [configure an existing server](https://sap.github.io/jenkins-library/infrastructure/customjenkins/). +The pipeline responsible for building ABAP add-ons has been created specifically for [Jenkins](https://www.jenkins.io). Therefore, a Jenkins Server is required. Please follow these instructions to [configure an existing server](https://sap.github.io/jenkins-library/infrastructure/customjenkins/). #### Git Repository diff --git a/documentation/docs/scenarios/gCTS_Scenario.md b/documentation/docs/scenarios/gCTS_Scenario.md index 1f957bc45e..394d990494 100644 --- a/documentation/docs/scenarios/gCTS_Scenario.md +++ b/documentation/docs/scenarios/gCTS_Scenario.md @@ -23,7 +23,7 @@ This scenario explains how to use a pipeline to deploy a commit to a test system - You have at least two ABAP systems with a version SAP S/4HANA 2020 or higher. You need one development system that you use to push objects to the Git repository, and a test system on which you run the pipeline. You have created and cloned the Git repository on all systems, on the development system with the *Development* role, and on the others with the *Provided* role. - You have enabled [ATC](https://help.sap.com/docs/ABAP_PLATFORM_NEW/ba879a6e2ea04d9bb94c7ccd7cdac446/62c41ad841554516bb06fb3620540e47.html) checks in transaction ATC in the test system. - You have access to a Jenkins instance including the [Warnings-Next-Generation Plugin](https://plugins.jenkins.io/warnings-ng/). The plug-in must be installed separately. It is required to view the results of the testing after the pipeline has run. - For the gCTS scenario, we recommend that you use the [Custom Jenkins setup](https://www.project-piper.io/infrastructure/customjenkins/) even though it is possible to run the gCTS scenario with [Piper´s CX server](https://www.project-piper.io/infrastructure/overview/). + For the gCTS scenario, we recommend that you use the [Custom Jenkins setup](https://www.project-piper.io/infrastructure/customjenkins/). - You have set up a suitable Jenkins instance as described under [Getting Started with Project "Piper"](https://www.project-piper.io/guidedtour/) under *Create Your First Pipeline*. - The user that is used for the execution of the pipeline must have the credentials entered in gCTS as described in the gCTS documentation under [Set User-Specific Authentication](https://help.sap.com/docs/ABAP_PLATFORM_NEW/4a368c163b08418890a406d413933ba7/3431ebd6fbf241778cd60587e7b5dc3e.html).