From c5744310000dbc4705e6191de55a5a69ff9d574c Mon Sep 17 00:00:00 2001 From: Vijai Kalathur Date: Wed, 13 May 2020 10:37:43 -0400 Subject: [PATCH 01/10] Update working-with-pipelines.md --- .../working-with-pipelines.md | 191 ++++++++++++------ 1 file changed, 124 insertions(+), 67 deletions(-) diff --git a/publish/working-with-pipelines/working-with-pipelines.md b/publish/working-with-pipelines/working-with-pipelines.md index 6391680..8ad86ca 100644 --- a/publish/working-with-pipelines/working-with-pipelines.md +++ b/publish/working-with-pipelines/working-with-pipelines.md @@ -32,32 +32,102 @@ guide-category: pipelines // --> -## Intro +# Intro -[Pipelines](https://github.com/tektoncd/pipeline/tree/master/docs#usage) enable a continuous input and continuous delivery (CI/CD) workflow. A set of default tasks and pipelines are provided that can be associated with application stacks. These pipelines validate the application stack is active, build the application stack, publish the image to a container registry, scan the published image, and then deploy the application to the Kubernetes cluster. You can also create your own tasks and pipelines and customize the pre-built pipelines and tasks. All tasks and pipelines are activated by the product operator. +[Pipelines](https://github.com/tektoncd/pipeline/tree/master/docs#usage) enable a continuous input and continuous delivery (CI/CD) workflow. A set of default tasks and pipelines are provided that can be associated with application stacks. These pipelines leverage steps & tasks that provide the following capabilities: +- build the application stack +- enforce goverance opolicy +- publish the image to a container registry +- scan the published image +- sign the image +- Retag an image +- deploy the application to the Kubernetes cluster +- promote service to a gitops repo (This feature is Tech Preview in this release) +- deploy the application using Kustomize (This feature is Tech Preview in this release) + +You can also create your own tasks and pipelines and customize the pre-built pipelines and tasks. All tasks and pipelines are activated by the product operator. To learn more about pipelines and creating new tasks, see [the pipeline tutorial](https://github.com/tektoncd/pipeline/blob/master/docs/tutorial.md). -## Default tasks and pipelines +## Kabanero tasks and pipelines + +A set of Kabanero tasks and pipelines are provided in the [Kabanero pipelines repository](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/). Details of some of the primary pipelines and tasks are described below. + +### Events pipelines + +The tasks & pipelines provided in the [Kabanero pipelines repository events directory](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/incubator/events) are geared to work with the Kabanero events operator. Please follow the instructions in to setup the organization webhook and EventMediator to drive these pipelines. + +There are 4 primary pipelines provided here to help illustrate the following work flow. + +* **Jane makes an update to the application and creates a new PR** + +This will trigger the `build-pl` which will build the application code and build the application image using the `build-task`. The PR will be updated with the results of the build pipeline. + +* **The PR is then merged into master** + +This will trigger the `build-push-promote-pl` which will enforce governance policy, build the code, optionally sign the image, push it to the image registry, scan the image, optionally deploy the image on the cluster, and optionally promote the service to the configured gitops repo. + +The pipeline invokes the following tasks to accomplish the steps listed: + * [build-push-promote-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/build-push-promote-task.yaml) + This task first does a pre-build goverance policy check to validate the stack version in the app repo is allowed to build based on the goverance policy that is configured. It then builds a container image from the artifacts in the git-source repository by using `appsody build`. The appsody build command leverages [Buildah](https://github.com/containers/buildah) to build the image. The command also generates the `app-deploy.yaml` that is used for deployment. If there is already a copy of the `app-deploy.yaml` file in the source repository, it is merged with the new one generated by this step. After the image is built, the image is then optionally signed if the necessary configuration is setup. Please refer to the [image signing operator](https://github.com/kabanero-io/kabanero-security/tree/master/pipelines/samples/signing-operator) for more information on configuring image signing. The image is then pushed to the configured image registry. + + (Tech preview feature) A configmap called `gitops-map` in the Kabanero namespace can optionally be configured to promote the service to a gitops repo after the build. The step will invoke the [`services promote`](https://github.com/rhd-gitops-example/services) command to create a PR with the updated `app-deploy.yaml` file in the configured gitops repo. The following key value pairs should be setup in the configmap: + ``` + kind: ConfigMap + apiVersion: v1 + metadata: + name: gitops-map + namespace: kabanero + data: + gitops-repo-enabled: + gitops-repo-url: + gitops-commit-user-name: + gitops-commit-user-email: + ``` + + A secret called `gitops-token` also has to be created in the Kabanero namspace. An example yaml is shown below to create the secret. + ``` + apiVersion: v1 + kind: Secret + metadata: + name: gitops-token + annotations: + tekton.dev/git-0: https://github.com + namespace: kabanero + type: kubernetes.io/basic-auth + stringData: + username: + password: + ``` + + * [deploy-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-task.yaml) + If the `webhooks-tekton-local-deploy` property is set to true in the mediator, the image is deployed to the namespace configured in the `app-deploy.yaml`. By default, the application will be deployed in the `kabanero` namespace. + + * [image-scan-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/image-scan-task.yaml) + The `image-scan-task` task will initiate a container scan of the image published by the `build-push-task` using OpenSCAP. The results of the scan are published in the logs of the task. -The default tasks and pipelines are provided in the [pipelines repository](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/incubator). Details of some of the primary pipelines and tasks are described in the following information. +* **A release of the application is created** +This will trigger the `image-retag-pl` pipeline which leverages the [image-retag-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/image-retag-task.yaml) to create a new tag of the image to match with the git release. -### The build, push and deploy pipeline +* **PR in Gitops repo is merged** (This feature is Tech Preview in this release) +When the PR that was created by the promote step of the `build-push-promote-pl` is merged in the gitops repo, it will trigger the `deploy-kustomize-pl` pipeline which leverages the [deploy-kustomize-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-kustomize-task.yaml) to trigger a deployment to the environment configured in the gitops repo. -- [build-deploy-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/build-deploy-pl.yaml) +### Legacy incubator pipelines (Deprecated in Kabanero 0.9.0) - This file is the primary pipeline that showcases all the tasks supplied. It validates that the application stack is active, builds the application stack, publishes the application image to the container registry, does a security scan of the image, and conditionally deploys the application. When running the pipeline via a webhook, the pipeline leverages the triggers functionality to conditionally deploy the application only when a pull request is merged in the git repo. Other actions that trigger the pipeline run, will validate, build, push, and scan the image. +The set of Kabanero tasks and pipelines are provided in the [Kabanero pipelines repository](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/incubator) are being deprecated. These tasks and pipelines were intended to illustrate work flows with the Tekton webhooks extenstion which is also being deprecated. Please use the events pipelines and tasks mentioned in the section above instead. -### Tasks +Details of some of the primary pipelines and tasks are described below. -- [validate-active-stack-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/validate-active-stack-task.yaml) +- [build-deploy-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/build-deploy-pl.yaml) -This task validates the stack is allowed to build and deploy on the cluster. It checks the digest of the stack image specified in the `.appsody-config.yaml` file of the project and validates that it matches the digest of the stack image that's active on the cluster. If the digests do not match, the pipeline fails and none of the other steps execute. +This is the primary pipeline that showcases a majority of the tasks supplied in the Kabanero pipelines incubator repo. It enforces governance policy, builds the code, optionally signs the image, pushes it to the image registry, scans the image, and will conditionally deploy the image on the cluster. When running the pipeline via a webhook, the pipeline leverages the triggers functionality to conditionally deploy the application only when a pull request is merged in the git repo. Other actions that trigger the pipeline run will enforce governance policy, build, push, and scan the image. - [build-push-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/build-push-task.yaml) - This file builds a container image from the artifacts in the git-source repository by using `appsody build`. The appsody build command leverages the [Buildah](https://github.com/containers/buildah) options. After the image is built, the image is published to the container registry that is configured. The build-push-task also generates the `app-deploy.yaml` that is used by the `deploy-task`. If there is already a copy of the `app-deploy.yaml` file in the source repository, it is merged with the new one generated by this step. + This file builds a container image from the artifacts in the git-source repository by using `appsody build`. The appsody build command leverages the [Buildah](https://github.com/containers/buildah) options. After the image is built, the image is published to the container registry that is configured. The build-push-task also generates the `app-deploy.yaml` that is used by the `deploy-task`. If there is already a copy of the `app-deploy.yaml` file in the source repository, it is merged with the new one generated by this step. + + To enable image signing, please refer to the [image signing operator](https://github.com/kabanero-io/kabanero-security/tree/master/pipelines/samples/signing-operator). - [deploy-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/deploy-task.yaml) @@ -66,10 +136,39 @@ This task validates the stack is allowed to build and deploy on the cluster. It - [image-scan-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/image-scan-task.yaml) The `image-scan-task` task will initiate a container scan of the image published by the `build-push-task` using OpenSCAP. The results of the scan are published in the logs of the task. + +For more tasks and pipelines, see [the kabanero-pipelines repo](https://github.com/kabanero-io/kabanero-pipelines). + +# Before running pipelines + +## Associating pipelines with applications stacks in Kabanero CRD + +### Events pipelines + +The pipelines can be associated with an application stack in the Kabanero custom resource definition (CRD). This is an example CRD: + +```yaml +apiVersion: kabanero.io/v1alpha1 +kind: Kabanero +metadata: + name: kabanero +spec: + version: "0.9.0" + stacks: + repositories: + - name: central + https: + url: https://github.com/kabanero-io/collections/releases/download/0.9.0/kabanero-index.yaml + pipelines: + - id: default + sha256: 14d59b7ebae113c18fb815c2ccfd8a846c5fbf91d926ae92e0017ca5caf67c95 + https: + url: https://github.com/kabanero-io/kabanero-pipelines/releases/download/0.9.0/kabanero-events-pipelines.tar.gz +``` -For more tasks and pipelines, see [the pipelines repo](https://github.com/kabanero-io/kabanero-pipelines). +When the Kabanero operator activates the CRD, it associates the pipelines in the pipelines archive with each of the stacks in the stack hub. The default pipelines are intended to work with all the stacks in the stack hub in the previous example. When the operator activates all the pipeline resources (such as the tasks, trigger bindings, and pipelines) in the archive, it will add a suffix to the name of the resource with the shorted Digest of the pipelines archive. This provides an easy way to have multiple versions of the same pipeline active on the cluster. -### Associating pipelines with applications stacks in Kabanero CRD +### Legacy pipelines The pipelines can be associated with an application stack in the Kabanero custom resource definition (CRD). This is an example CRD: @@ -95,9 +194,9 @@ spec: When the product operator activates the CRD, it associates the pipelines in the pipelines archive with each of the stacks in the stack hub. The default pipelines are intended to work with all the stacks in the stack hub in the previous example. All of the pipeline-related resources (such as the tasks, trigger bindings, and pipelines) prefix the name of the resource with the keyword `StackId`. When the operator activates these resources, it replaces the keyword with the name of the stack it is activating. -### Creating and updating tasks and pipelines +## Creating and updating your own tasks and pipelines -The default tasks and pipelines can be updated by forking the pipelines repo and editing the files under `pipelines/incubator`. The easiest way to generate the archive for use by the Kabanero CRD is to run the [package.sh](https://github.com/kabanero-io/kabanero-pipelines/blob/master/ci/package.sh) script. The script generates the archive file with the necessary pipeline artifacts and a `manifest.yaml` file that describes the contents of the archive. Copy the `package.sh` file to the root directory of your pipelines project and run it. It generates the pipelines archive file under `ci/assests`. +The default tasks and pipelines can be updated by forking the Kabanero Pipelines repo and editing the files under `pipelines/`. The easiest way to generate the archive for use by the Kabanero CRD is to run the [package.sh](https://github.com/kabanero-io/kabanero-pipelines/blob/master/ci/package.sh) script from the root directory of the pipelines project. The script generates the archive files with the necessary pipeline artifacts and a `manifest.yaml` file that describes the contents of the archive. It generates the pipelines archive file under `ci/assests`. It generates seperate archives for the legacy incubator pipelines, events pipelines, and the experimental gitops pipleines. Alternatively, you can run the Travis build against a release of your pipelines repo, which also generates the archive file with a `manifest.yaml` file and attaches it to your release. @@ -109,49 +208,9 @@ For more detailed instructions, see [Curating Pipelines](../curating-pipelines/c // ================================================================================================= --> -## Using stacks that are published to internal and private registries in pipelines +## Using stacks published to internal and private registries in pipelines -If you are publishing your application stack images to any registry other than Docker hub, you can specify your custom registry when you initialize a stack by using the `--stack-registry` option in the `appsody init` command. Specifying a custom registry updates the stack name in the `.appsody-config.yaml` to include the registry information that is consumed by the pipeline. - -Alternatively, you can use a configmap to configure the custom repository from which your pipelines pulls the container images. - -1. After you clone the `kabanero-pipelines` repository, find the `stack-image-registry-map.yaml` configmap template file. Add your container registry URL to this file in place of the `default-stack-image-registry-url` statement. - - ```shell - cd kabanero-pipelines/pipelines/sample-helper-files/ - vi stack-image-registry-map.yaml - ``` - -1. If your custom application stack image is stored in an internal OpenShift registry, the service account that is associated with the pipelines must be configured to allow the pipelines to pull from the internal registry without configuring a secret. If your custom application stack is stored in a container registry with an external route, follow these steps to set up a Kubernetes secret: - - - Find the `default-stack-image-registry-secret.yaml` template file in the cloned kabanero-pipelines repo (`kabanero-pipelines/pipelines/sample-helper-files/`) and update it with the username and token password for the container registry URL you specified previously. - - - Create a Base64 format version of the username and password for the external route container registry URL. - - ```shell - echo -n | base64 - echo -n | base64 - ``` - - - Update the `default-stack-image-registry-secret.yaml` file with the Base64 formatted username and password. - - ```shell - vi default-stack-image-registry-secret.yaml - ``` - - - Apply the `default-stack-image-registry-secret.yaml` file to the cluster - - ```shell - oc apply -f default-stack-image-registry-secret.yaml - ``` - -1. Apply the following configmap file, which will set your container registry. - - ```shell - oc apply -f stack-image-registry-map.yaml - ``` - -**NOTE:** If a value is specified in both the config map and in the `.appsody-config.yaml` and they are different, the config map takes precedence. +If you are publishing your application stack images to any registry other than Docker hub, you can specify your custom registry when you initialize a stack by using the `--stack-registry` option in the `appsody init` command. Specifying a custom registry updates the stack name in the `.appsody-config.yaml` to include the registry information that is consumed by the pipeline. -## Running pipelines +# Running pipelines Explore how to use pipelines to build and manage application stacks. -### Prerequisites +## Prerequisites 1. [Kabanero foundation](https://github.com/kabanero-io/kabanero-foundation) must be installed on a supported Kubernetes deployment. -1. [A pipelines dashboard](https://github.com/tektoncd/dashboard) is installed by default with the product operator. To find the pipelines dashboard URL, login to your cluster and run the `oc get routes` command or in the product landing page. - 1. A persistent volume must be configured. See the following section for details. 1. Secrets for the git repo (if private) and image repository -### Getting started +## Getting started Follow these steps: @@ -285,11 +342,11 @@ Follow these steps: Alternatively, you can [configure secrets in the Kubernetes console or set them up by using the Kubernetes CLI](https://docs.okd.io/latest/dev_guide/secrets.html#creating-secrets). -### Running pipelines by using the pipelines dashboard webhook extension +## Running pipelines by using the pipelines dashboard webhook extension (Deprecated) You can use the [pipelines dashboard webhook extension](https://github.com/tektoncd/experimental/blob/master/webhooks-extension/docs/GettingStarted.md) to drive pipelines that automatically build and deploy an application whenever you update the code in your GitHub repo. Events such as commits or pull requests can be set up to automatically trigger pipeline runs. -### Running pipelines by using a script +## Running pipelines by using a script If you are developing a new pipeline and want to test it in a tight loop, you might want to use a script or manually drive the pipeline. @@ -323,7 +380,7 @@ If you are developing a new pipeline and want to test it in a tight loop, you mi ./manual-pipeline-run-script.sh -r https://github.com/mygitid/appsody-test-project -i docker-registry.default.svc:5000/kabanero/my-java-openliberty-image -c java-openliberty" ``` -### Running pipelines manually from the command line +## Running pipelines manually from the command line Follow these steps to run a pipeline directly from the command line: @@ -350,7 +407,7 @@ Follow these steps to run a pipeline directly from the command line: oc apply -f -pipeline-resources.yaml ``` -### Activating tasks and pipelines +## Activating tasks and pipelines The installations that activate the featured application stacks also activate the tasks and pipelines. If you are creating a new task or pipeline, activate it manually, as shown in the following example. @@ -359,7 +416,7 @@ oc apply -f oc apply -f ``` -### Running the pipeline +## Running the pipeline A sample `manual-pipeline-run-template.yaml` file is provided in the [/pipelines/sample-helper-files](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/sample-helper-files) directory. Rename the template file to a name of your choice (for example, pipeline-run.yaml), and update the file to replace `application-stack-name` with the name of your application stack. After you update the file, run it as shown in the following example. From fba976031e4f25405225ef0660f67683a5a6df8c Mon Sep 17 00:00:00 2001 From: Jeff Antley Date: Wed, 13 May 2020 11:06:09 -0400 Subject: [PATCH 02/10] Update working-with-pipelines.md --- .../working-with-pipelines.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/publish/working-with-pipelines/working-with-pipelines.md b/publish/working-with-pipelines/working-with-pipelines.md index 8ad86ca..f9d1fd1 100644 --- a/publish/working-with-pipelines/working-with-pipelines.md +++ b/publish/working-with-pipelines/working-with-pipelines.md @@ -34,15 +34,15 @@ guide-category: pipelines # Intro -[Pipelines](https://github.com/tektoncd/pipeline/tree/master/docs#usage) enable a continuous input and continuous delivery (CI/CD) workflow. A set of default tasks and pipelines are provided that can be associated with application stacks. These pipelines leverage steps & tasks that provide the following capabilities: +[Pipelines](https://github.com/tektoncd/pipeline/tree/master/docs#usage) enable a continuous input and continuous delivery (CI/CD) workflow. A set of default tasks and pipelines are provided that can be associated with application stacks. These pipelines leverage steps and tasks that provide the following capabilities: - build the application stack -- enforce goverance opolicy +- enforce the goverance policy - publish the image to a container registry - scan the published image - sign the image -- Retag an image +- retag an image - deploy the application to the Kubernetes cluster -- promote service to a gitops repo (This feature is Tech Preview in this release) +- promote a service to a gitops repo (This feature is Tech Preview in this release) - deploy the application using Kustomize (This feature is Tech Preview in this release) You can also create your own tasks and pipelines and customize the pre-built pipelines and tasks. All tasks and pipelines are activated by the product operator. @@ -51,25 +51,25 @@ To learn more about pipelines and creating new tasks, see [the pipeline tutorial ## Kabanero tasks and pipelines -A set of Kabanero tasks and pipelines are provided in the [Kabanero pipelines repository](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/). Details of some of the primary pipelines and tasks are described below. +A set of Kabanero tasks and pipelines are provided in the [Kabanero pipelines repository](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/). Details of some of the primary pipelines and tasks are described below. ### Events pipelines -The tasks & pipelines provided in the [Kabanero pipelines repository events directory](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/incubator/events) are geared to work with the Kabanero events operator. Please follow the instructions in to setup the organization webhook and EventMediator to drive these pipelines. +The tasks and pipelines provided in the [Kabanero pipelines repository events directory](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/incubator/events) are geared to work with the Kabanero events operator. Follow the instructions in to set up the organization webhook and EventMediator to drive these pipelines. -There are 4 primary pipelines provided here to help illustrate the following work flow. +There are four primary pipelines provided here to help illustrate the following work flow. -* **Jane makes an update to the application and creates a new PR** +* **Jane makes an update to the application and creates a new pull request** -This will trigger the `build-pl` which will build the application code and build the application image using the `build-task`. The PR will be updated with the results of the build pipeline. +This action triggers the `build-pl` pipeline which builds the application code and builds the application image using the `build-task`. The pull request (PR) is updated with the results of the build pipeline. -* **The PR is then merged into master** +* **The pull request is then merged into the master branch** -This will trigger the `build-push-promote-pl` which will enforce governance policy, build the code, optionally sign the image, push it to the image registry, scan the image, optionally deploy the image on the cluster, and optionally promote the service to the configured gitops repo. +This action triggers the `build-push-promote-pl` pipeline which enforces the governance policy, builds the code, optionally signs the image, pushes it to the image registry, scans the image, optionally deploys the image on the cluster, and optionally promotes the service to the configured gitops repository. The pipeline invokes the following tasks to accomplish the steps listed: * [build-push-promote-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/build-push-promote-task.yaml) - This task first does a pre-build goverance policy check to validate the stack version in the app repo is allowed to build based on the goverance policy that is configured. It then builds a container image from the artifacts in the git-source repository by using `appsody build`. The appsody build command leverages [Buildah](https://github.com/containers/buildah) to build the image. The command also generates the `app-deploy.yaml` that is used for deployment. If there is already a copy of the `app-deploy.yaml` file in the source repository, it is merged with the new one generated by this step. After the image is built, the image is then optionally signed if the necessary configuration is setup. Please refer to the [image signing operator](https://github.com/kabanero-io/kabanero-security/tree/master/pipelines/samples/signing-operator) for more information on configuring image signing. The image is then pushed to the configured image registry. + This task first does a pre-build goverance policy check to validate the stack version in the application repository is allowed to build based on the goverance policy that is configured. It then builds a container image from the artifacts in the git-source repository by using `appsody build`. The appsody build command leverages [Buildah](https://github.com/containers/buildah) to build the image. The command also generates the `app-deploy.yaml` that is used for deployment. If there is already a copy of the `app-deploy.yaml` file in the source repository, it is merged with the new one generated by this step. After the image is built, the image is then optionally signed if the necessary configuration is setup. Refer to the [image signing operator](https://github.com/kabanero-io/kabanero-security/tree/master/pipelines/samples/signing-operator) for more information on configuring image signing. The image is then pushed to the configured image registry. (Tech preview feature) A configmap called `gitops-map` in the Kabanero namespace can optionally be configured to promote the service to a gitops repo after the build. The step will invoke the [`services promote`](https://github.com/rhd-gitops-example/services) command to create a PR with the updated `app-deploy.yaml` file in the configured gitops repo. The following key value pairs should be setup in the configmap: ``` From 7e0d4e13d0c801643d13e74c19ab5515acd51379 Mon Sep 17 00:00:00 2001 From: Jeff Antley Date: Wed, 13 May 2020 11:19:09 -0400 Subject: [PATCH 03/10] Update working-with-pipelines.md --- .../working-with-pipelines.md | 30 +++++++++---------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/publish/working-with-pipelines/working-with-pipelines.md b/publish/working-with-pipelines/working-with-pipelines.md index f9d1fd1..53dcade 100644 --- a/publish/working-with-pipelines/working-with-pipelines.md +++ b/publish/working-with-pipelines/working-with-pipelines.md @@ -71,7 +71,7 @@ The pipeline invokes the following tasks to accomplish the steps listed: * [build-push-promote-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/build-push-promote-task.yaml) This task first does a pre-build goverance policy check to validate the stack version in the application repository is allowed to build based on the goverance policy that is configured. It then builds a container image from the artifacts in the git-source repository by using `appsody build`. The appsody build command leverages [Buildah](https://github.com/containers/buildah) to build the image. The command also generates the `app-deploy.yaml` that is used for deployment. If there is already a copy of the `app-deploy.yaml` file in the source repository, it is merged with the new one generated by this step. After the image is built, the image is then optionally signed if the necessary configuration is setup. Refer to the [image signing operator](https://github.com/kabanero-io/kabanero-security/tree/master/pipelines/samples/signing-operator) for more information on configuring image signing. The image is then pushed to the configured image registry. - (Tech preview feature) A configmap called `gitops-map` in the Kabanero namespace can optionally be configured to promote the service to a gitops repo after the build. The step will invoke the [`services promote`](https://github.com/rhd-gitops-example/services) command to create a PR with the updated `app-deploy.yaml` file in the configured gitops repo. The following key value pairs should be setup in the configmap: + (Tech preview feature) A configmap called `gitops-map` in the Kabanero namespace can optionally be configured to promote the service to a gitops repo after the build. The step will invoke the [`services promote`](https://github.com/rhd-gitops-example/services) command to create a PR with the updated `app-deploy.yaml` file in the configured gitops repo. The following key value pairs should be setup in the configmap: ``` kind: ConfigMap apiVersion: v1 @@ -85,7 +85,7 @@ The pipeline invokes the following tasks to accomplish the steps listed: gitops-commit-user-email: ``` - A secret called `gitops-token` also has to be created in the Kabanero namspace. An example yaml is shown below to create the secret. + A secret called `gitops-token` also has to be created in the Kabanero namspace. This example yaml creates the secret. ``` apiVersion: v1 kind: Secret @@ -101,7 +101,7 @@ The pipeline invokes the following tasks to accomplish the steps listed: ``` * [deploy-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-task.yaml) - If the `webhooks-tekton-local-deploy` property is set to true in the mediator, the image is deployed to the namespace configured in the `app-deploy.yaml`. By default, the application will be deployed in the `kabanero` namespace. + If the `webhooks-tekton-local-deploy` property is set to `true` in the mediator, the image is deployed to the namespace configured in the `app-deploy.yaml`. By default, the application will be deployed in the `kabanero` namespace. * [image-scan-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/image-scan-task.yaml) The `image-scan-task` task will initiate a container scan of the image published by the `build-push-task` using OpenSCAP. The results of the scan are published in the logs of the task. @@ -110,22 +110,22 @@ The pipeline invokes the following tasks to accomplish the steps listed: This will trigger the `image-retag-pl` pipeline which leverages the [image-retag-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/image-retag-task.yaml) to create a new tag of the image to match with the git release. -* **PR in Gitops repo is merged** (This feature is Tech Preview in this release) -When the PR that was created by the promote step of the `build-push-promote-pl` is merged in the gitops repo, it will trigger the `deploy-kustomize-pl` pipeline which leverages the [deploy-kustomize-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-kustomize-task.yaml) to trigger a deployment to the environment configured in the gitops repo. +* **The pull request in the gitops repo is merged** (This feature is Tech Preview in this release) +When the PR that was created by the promote step of the `build-push-promote-pl` is merged in the gitops repo, it will trigger the `deploy-kustomize-pl` pipeline, which leverages the [deploy-kustomize-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-kustomize-task.yaml) to trigger a deployment to the environment configured in the gitops repo. ### Legacy incubator pipelines (Deprecated in Kabanero 0.9.0) -The set of Kabanero tasks and pipelines are provided in the [Kabanero pipelines repository](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/incubator) are being deprecated. These tasks and pipelines were intended to illustrate work flows with the Tekton webhooks extenstion which is also being deprecated. Please use the events pipelines and tasks mentioned in the section above instead. +The set of Kabanero tasks and pipelines are provided in the [Kabanero pipelines repository](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/incubator) are deprecated in Kabanero 0.9.0. These tasks and pipelines illustrate work flows with the Tekton webhooks extension, which is also now deprecated. Use the new events pipelines and tasks instead. -Details of some of the primary pipelines and tasks are described below. +Details of some of the primary pipelines and tasks: - [build-deploy-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/build-deploy-pl.yaml) -This is the primary pipeline that showcases a majority of the tasks supplied in the Kabanero pipelines incubator repo. It enforces governance policy, builds the code, optionally signs the image, pushes it to the image registry, scans the image, and will conditionally deploy the image on the cluster. When running the pipeline via a webhook, the pipeline leverages the triggers functionality to conditionally deploy the application only when a pull request is merged in the git repo. Other actions that trigger the pipeline run will enforce governance policy, build, push, and scan the image. +This is the primary pipeline that showcases a majority of the tasks supplied in the Kabanero pipelines incubator repo. It enforces governance policy, builds the code, optionally signs the image, pushes it to the image registry, scans the image, and can conditionally deploy the image on the cluster. When running the pipeline via a webhook, the pipeline leverages the triggers functionality to conditionally deploy the application only when a pull request is merged in the git repo. Other actions that trigger the pipeline run enforce governance policy, build, push, and scan the image. - [build-push-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/build-push-task.yaml) - This file builds a container image from the artifacts in the git-source repository by using `appsody build`. The appsody build command leverages the [Buildah](https://github.com/containers/buildah) options. After the image is built, the image is published to the container registry that is configured. The build-push-task also generates the `app-deploy.yaml` that is used by the `deploy-task`. If there is already a copy of the `app-deploy.yaml` file in the source repository, it is merged with the new one generated by this step. + This file builds a container image from the artifacts in the git-source repository by using `appsody build`. The appsody build command leverages the [Buildah](https://github.com/containers/buildah) options. After the image is built, the image is published to the configured container registry. The build-push-task also generates the `app-deploy.yaml` that is used by the `deploy-task`. If there is already a copy of the `app-deploy.yaml` file in the source repository, it is merged with the new one generated by this step. To enable image signing, please refer to the [image signing operator](https://github.com/kabanero-io/kabanero-security/tree/master/pipelines/samples/signing-operator). @@ -135,17 +135,17 @@ This is the primary pipeline that showcases a majority of the tasks supplied in - [image-scan-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/image-scan-task.yaml) - The `image-scan-task` task will initiate a container scan of the image published by the `build-push-task` using OpenSCAP. The results of the scan are published in the logs of the task. + The `image-scan-task` task initiates a container scan of the image published by the `build-push-task` using OpenSCAP. The results of the scan are published in the logs of the task. For more tasks and pipelines, see [the kabanero-pipelines repo](https://github.com/kabanero-io/kabanero-pipelines). # Before running pipelines -## Associating pipelines with applications stacks in Kabanero CRD +## Associating pipelines with applications stacks in Kabanero custom resource ### Events pipelines -The pipelines can be associated with an application stack in the Kabanero custom resource definition (CRD). This is an example CRD: +The pipelines can be associated with an application stack in the Kabanero custom resource definition (CRD). For example: ```yaml apiVersion: kabanero.io/v1alpha1 @@ -166,11 +166,11 @@ spec: url: https://github.com/kabanero-io/kabanero-pipelines/releases/download/0.9.0/kabanero-events-pipelines.tar.gz ``` -When the Kabanero operator activates the CRD, it associates the pipelines in the pipelines archive with each of the stacks in the stack hub. The default pipelines are intended to work with all the stacks in the stack hub in the previous example. When the operator activates all the pipeline resources (such as the tasks, trigger bindings, and pipelines) in the archive, it will add a suffix to the name of the resource with the shorted Digest of the pipelines archive. This provides an easy way to have multiple versions of the same pipeline active on the cluster. +When the product operator activates the Kabanero CRD, it associates the pipelines in the pipelines archive with each of the stacks in the stack hub. The default pipelines are intended to work with all the stacks in the stack hub in the previous example. When the operator activates all the pipeline resources (such as the tasks, trigger bindings, and pipelines) in the archive, it will add a suffix to the name of the resource with the shorted digest of the pipelines archive. This provides an easy way to have multiple versions of the same pipeline active on the cluster. ### Legacy pipelines -The pipelines can be associated with an application stack in the Kabanero custom resource definition (CRD). This is an example CRD: +The pipelines can be associated with an application stack in the Kabanero CRD. For example: ```yaml apiVersion: kabanero.io/v1alpha2 @@ -196,7 +196,7 @@ When the product operator activates the CRD, it associates the pipelines in the ## Creating and updating your own tasks and pipelines -The default tasks and pipelines can be updated by forking the Kabanero Pipelines repo and editing the files under `pipelines/`. The easiest way to generate the archive for use by the Kabanero CRD is to run the [package.sh](https://github.com/kabanero-io/kabanero-pipelines/blob/master/ci/package.sh) script from the root directory of the pipelines project. The script generates the archive files with the necessary pipeline artifacts and a `manifest.yaml` file that describes the contents of the archive. It generates the pipelines archive file under `ci/assests`. It generates seperate archives for the legacy incubator pipelines, events pipelines, and the experimental gitops pipleines. +The default tasks and pipelines can be updated by forking the Kabanero Pipelines repo and editing the files under `pipelines/`. The easiest way to generate the archive for use by the Kabanero CRD is to run the [package.sh](https://github.com/kabanero-io/kabanero-pipelines/blob/master/ci/package.sh) script from the root directory of the pipelines project. The script generates the archive files with the necessary pipeline artifacts and a `manifest.yaml` file that describes the contents of the archive. It generates the pipelines archive file under `ci/assests`. It generates seperate archives for the legacy incubator pipelines, events pipelines, and the experimental gitops pipleines. Alternatively, you can run the Travis build against a release of your pipelines repo, which also generates the archive file with a `manifest.yaml` file and attaches it to your release. From f84ea057ce7b78904f816d78287063901bf25882 Mon Sep 17 00:00:00 2001 From: Jeff Antley Date: Wed, 13 May 2020 11:19:43 -0400 Subject: [PATCH 04/10] Update working-with-pipelines.md --- publish/working-with-pipelines/working-with-pipelines.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/publish/working-with-pipelines/working-with-pipelines.md b/publish/working-with-pipelines/working-with-pipelines.md index 53dcade..d8c45ce 100644 --- a/publish/working-with-pipelines/working-with-pipelines.md +++ b/publish/working-with-pipelines/working-with-pipelines.md @@ -32,7 +32,7 @@ guide-category: pipelines // --> -# Intro +# Introduction [Pipelines](https://github.com/tektoncd/pipeline/tree/master/docs#usage) enable a continuous input and continuous delivery (CI/CD) workflow. A set of default tasks and pipelines are provided that can be associated with application stacks. These pipelines leverage steps and tasks that provide the following capabilities: - build the application stack From ca1d5636cd540cd92c8b602d544edce053a05425 Mon Sep 17 00:00:00 2001 From: Vijai Kalathur Date: Wed, 13 May 2020 11:31:17 -0400 Subject: [PATCH 05/10] Update working-with-pipelines.md --- publish/working-with-pipelines/working-with-pipelines.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/publish/working-with-pipelines/working-with-pipelines.md b/publish/working-with-pipelines/working-with-pipelines.md index d8c45ce..6206814 100644 --- a/publish/working-with-pipelines/working-with-pipelines.md +++ b/publish/working-with-pipelines/working-with-pipelines.md @@ -79,8 +79,8 @@ The pipeline invokes the following tasks to accomplish the steps listed: name: gitops-map namespace: kabanero data: - gitops-repo-enabled: - gitops-repo-url: + gitops-repository-url: + gitops-repository-type: gitops-commit-user-name: gitops-commit-user-email: ``` From a455a08109a6d9ba82158a9e716dddabda70d01e Mon Sep 17 00:00:00 2001 From: Jeff Antley Date: Fri, 15 May 2020 12:02:46 -0400 Subject: [PATCH 06/10] Update working-with-pipelines.md --- publish/working-with-pipelines/working-with-pipelines.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/publish/working-with-pipelines/working-with-pipelines.md b/publish/working-with-pipelines/working-with-pipelines.md index 6206814..1848d39 100644 --- a/publish/working-with-pipelines/working-with-pipelines.md +++ b/publish/working-with-pipelines/working-with-pipelines.md @@ -59,7 +59,7 @@ The tasks and pipelines provided in the [Kabanero pipelines repository events di There are four primary pipelines provided here to help illustrate the following work flow. -* **Jane makes an update to the application and creates a new pull request** +* **A developer makes an update to the application and creates a new pull request** This action triggers the `build-pl` pipeline which builds the application code and builds the application image using the `build-task`. The pull request (PR) is updated with the results of the build pipeline. From f91d22b75e62789e49215ce92074ae4e4eaa84e2 Mon Sep 17 00:00:00 2001 From: Vijai Kalathur Date: Fri, 15 May 2020 12:13:23 -0400 Subject: [PATCH 07/10] Update working-with-pipelines.md --- .../working-with-pipelines.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/publish/working-with-pipelines/working-with-pipelines.md b/publish/working-with-pipelines/working-with-pipelines.md index 1848d39..3999a35 100644 --- a/publish/working-with-pipelines/working-with-pipelines.md +++ b/publish/working-with-pipelines/working-with-pipelines.md @@ -34,16 +34,16 @@ guide-category: pipelines # Introduction -[Pipelines](https://github.com/tektoncd/pipeline/tree/master/docs#usage) enable a continuous input and continuous delivery (CI/CD) workflow. A set of default tasks and pipelines are provided that can be associated with application stacks. These pipelines leverage steps and tasks that provide the following capabilities: +[Pipelines](https://github.com/tektoncd/pipeline/tree/master/docs#usage) enable a continuous integration and continuous delivery (CI/CD) workflow. A set of default tasks and pipelines are provided that can be associated with application stacks. These pipelines leverage steps and tasks that provide the following capabilities: - build the application stack -- enforce the goverance policy +- enforce the governance policy - publish the image to a container registry - scan the published image - sign the image - retag an image - deploy the application to the Kubernetes cluster -- promote a service to a gitops repo (This feature is Tech Preview in this release) -- deploy the application using Kustomize (This feature is Tech Preview in this release) +- promote a service to a GitOps repo (This feature is Tech Preview in this release) +- deploy a complete microservice scenario using Kustomize (This feature is Tech Preview in this release) You can also create your own tasks and pipelines and customize the pre-built pipelines and tasks. All tasks and pipelines are activated by the product operator. @@ -65,13 +65,13 @@ This action triggers the `build-pl` pipeline which builds the application code a * **The pull request is then merged into the master branch** -This action triggers the `build-push-promote-pl` pipeline which enforces the governance policy, builds the code, optionally signs the image, pushes it to the image registry, scans the image, optionally deploys the image on the cluster, and optionally promotes the service to the configured gitops repository. +This action triggers the `build-push-promote-pl` pipeline which enforces the governance policy, builds the code, optionally signs the image, pushes it to the image registry, scans the image, optionally deploys the image on the cluster, and optionally promotes the service to the configured GitOps repository. The pipeline invokes the following tasks to accomplish the steps listed: * [build-push-promote-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/build-push-promote-task.yaml) This task first does a pre-build goverance policy check to validate the stack version in the application repository is allowed to build based on the goverance policy that is configured. It then builds a container image from the artifacts in the git-source repository by using `appsody build`. The appsody build command leverages [Buildah](https://github.com/containers/buildah) to build the image. The command also generates the `app-deploy.yaml` that is used for deployment. If there is already a copy of the `app-deploy.yaml` file in the source repository, it is merged with the new one generated by this step. After the image is built, the image is then optionally signed if the necessary configuration is setup. Refer to the [image signing operator](https://github.com/kabanero-io/kabanero-security/tree/master/pipelines/samples/signing-operator) for more information on configuring image signing. The image is then pushed to the configured image registry. - (Tech preview feature) A configmap called `gitops-map` in the Kabanero namespace can optionally be configured to promote the service to a gitops repo after the build. The step will invoke the [`services promote`](https://github.com/rhd-gitops-example/services) command to create a PR with the updated `app-deploy.yaml` file in the configured gitops repo. The following key value pairs should be setup in the configmap: + (Tech preview feature) A configmap called `gitops-map` in the Kabanero namespace can optionally be configured to promote the service to a GitOps repo after the build. The step will invoke the [`services promote`](https://github.com/rhd-gitops-example/services) command to create a PR with the updated `app-deploy.yaml` file in the configured GitOps repo. The following key value pairs should be setup in the configmap: ``` kind: ConfigMap apiVersion: v1 @@ -113,9 +113,9 @@ This will trigger the `image-retag-pl` pipeline which leverages the [image-retag * **The pull request in the gitops repo is merged** (This feature is Tech Preview in this release) When the PR that was created by the promote step of the `build-push-promote-pl` is merged in the gitops repo, it will trigger the `deploy-kustomize-pl` pipeline, which leverages the [deploy-kustomize-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-kustomize-task.yaml) to trigger a deployment to the environment configured in the gitops repo. -### Legacy incubator pipelines (Deprecated in Kabanero 0.9.0) +### Incubator pipelines -The set of Kabanero tasks and pipelines are provided in the [Kabanero pipelines repository](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/incubator) are deprecated in Kabanero 0.9.0. These tasks and pipelines illustrate work flows with the Tekton webhooks extension, which is also now deprecated. Use the new events pipelines and tasks instead. +The set of Kabanero tasks and pipelines are provided in the [Kabanero pipelines repository](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/incubator) illustrate work flows with the Tekton webhooks extension. Details of some of the primary pipelines and tasks: @@ -168,7 +168,7 @@ spec: When the product operator activates the Kabanero CRD, it associates the pipelines in the pipelines archive with each of the stacks in the stack hub. The default pipelines are intended to work with all the stacks in the stack hub in the previous example. When the operator activates all the pipeline resources (such as the tasks, trigger bindings, and pipelines) in the archive, it will add a suffix to the name of the resource with the shorted digest of the pipelines archive. This provides an easy way to have multiple versions of the same pipeline active on the cluster. -### Legacy pipelines +### Incubator pipelines The pipelines can be associated with an application stack in the Kabanero CRD. For example: From 9b7e1630bc60d02b91d4ab0fd9dcda47d9e3dc5d Mon Sep 17 00:00:00 2001 From: Vijai Kalathur Date: Fri, 15 May 2020 14:07:50 -0400 Subject: [PATCH 08/10] Update working-with-pipelines.md --- .../working-with-pipelines.md | 62 ++++++++++++++++--- 1 file changed, 53 insertions(+), 9 deletions(-) diff --git a/publish/working-with-pipelines/working-with-pipelines.md b/publish/working-with-pipelines/working-with-pipelines.md index 3999a35..2d57dc1 100644 --- a/publish/working-with-pipelines/working-with-pipelines.md +++ b/publish/working-with-pipelines/working-with-pipelines.md @@ -61,11 +61,11 @@ There are four primary pipelines provided here to help illustrate the following * **A developer makes an update to the application and creates a new pull request** -This action triggers the `build-pl` pipeline which builds the application code and builds the application image using the `build-task`. The pull request (PR) is updated with the results of the build pipeline. +This action triggers the [build-pl](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/build-pl.yaml) pipeline which builds the application code and builds the application image using the [build-task](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/build-task.yaml). The pull request (PR) is updated with the results of the build pipeline. * **The pull request is then merged into the master branch** -This action triggers the `build-push-promote-pl` pipeline which enforces the governance policy, builds the code, optionally signs the image, pushes it to the image registry, scans the image, optionally deploys the image on the cluster, and optionally promotes the service to the configured GitOps repository. +This action triggers the [build-push-promote-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/build-push-promote-pl.yaml) pipeline which enforces the governance policy, builds the code, optionally signs the image, pushes it to the image registry, scans the image, optionally deploys the image on the cluster, and optionally promotes the service to the configured GitOps repository. The pipeline invokes the following tasks to accomplish the steps listed: * [build-push-promote-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/build-push-promote-task.yaml) @@ -108,14 +108,14 @@ The pipeline invokes the following tasks to accomplish the steps listed: * **A release of the application is created** -This will trigger the `image-retag-pl` pipeline which leverages the [image-retag-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/image-retag-task.yaml) to create a new tag of the image to match with the git release. +This will trigger the [image-retag-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/image-retag-pl.yaml) pipeline which leverages the [image-retag-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/image-retag-task.yaml) to create a new tag of the image to match with the git release. * **The pull request in the gitops repo is merged** (This feature is Tech Preview in this release) -When the PR that was created by the promote step of the `build-push-promote-pl` is merged in the gitops repo, it will trigger the `deploy-kustomize-pl` pipeline, which leverages the [deploy-kustomize-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-kustomize-task.yaml) to trigger a deployment to the environment configured in the gitops repo. +When the PR that was created by the promote step of the `build-push-promote-pl` is merged in the gitops repo, it will trigger the [deploy-kustomize-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-kustomize-pl.yaml) pipeline, which leverages the [deploy-kustomize-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-kustomize-task.yaml) to trigger a deployment to the environment configured in the gitops repo. ### Incubator pipelines -The set of Kabanero tasks and pipelines are provided in the [Kabanero pipelines repository](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/incubator) illustrate work flows with the Tekton webhooks extension. +The set of Kabanero tasks and pipelines are provided in the [Kabanero pipelines repository](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/incubator) illustrate work flows that work best with the Tekton webhooks extension. Details of some of the primary pipelines and tasks: @@ -139,6 +139,50 @@ This is the primary pipeline that showcases a majority of the tasks supplied in For more tasks and pipelines, see [the kabanero-pipelines repo](https://github.com/kabanero-io/kabanero-pipelines). +### Experimental GitOps pipelines (Tech Preview) + +There are two pipelines in the the experimental GitOps directory that demonstrate GitOps workflows. + +The [build-push-promote-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/experimental/gitops/build-push-promote-pl.yaml) pipeline enforces the governance policy, builds the code, optionally signs the image, pushes it to the image registry, scans the image, and optionally promotes the service to the configured GitOps repository. + +The pipeline invokes the following tasks to accomplish the steps listed: + * [build-push-promote-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/experimental/gitops/build-push-promote-task.yaml) + This task first does a pre-build goverance policy check to validate the stack version in the application repository is allowed to build based on the goverance policy that is configured. It then builds a container image from the artifacts in the git-source repository by using `appsody build`. The appsody build command leverages [Buildah](https://github.com/containers/buildah) to build the image. The image is then optionally signed and pushed to the configured image registry. + + A configmap called `gitops-map` in the Kabanero namespace can optionally be configured to promote the service to a GitOps repo after the build. The step will invoke the [`services promote`](https://github.com/rhd-gitops-example/services) command to create a PR with the updated `app-deploy.yaml` file in the configured GitOps repo. The following key value pairs should be setup in the configmap: + ``` + kind: ConfigMap + apiVersion: v1 + metadata: + name: gitops-map + namespace: kabanero + data: + gitops-repository-url: + gitops-repository-type: + gitops-commit-user-name: + gitops-commit-user-email: + ``` + + A secret called `gitops-token` also has to be created in the Kabanero namspace. This example yaml creates the secret. + ``` + apiVersion: v1 + kind: Secret + metadata: + name: gitops-token + annotations: + tekton.dev/git-0: https://github.com + namespace: kabanero + type: kubernetes.io/basic-auth + stringData: + username: + password: + ``` + * [image-scan-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/experimental/gitops/image-scan-task.yaml) + + The `image-scan-task` task initiates a container scan of the image published by the `build-push-task` using OpenSCAP. The results of the scan are published in the logs of the task. + +The [deploy-kustomize-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/experimental/gitops/deploy-kustomize-pl.yaml) pipeline, which leverages the [deploy-kustomize-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/experimental/gitops/deploy-kustomize-task.yaml) to trigger a deployment to the environment configured in the GitOps repo. + # Before running pipelines ## Associating pipelines with applications stacks in Kabanero custom resource @@ -166,7 +210,7 @@ spec: url: https://github.com/kabanero-io/kabanero-pipelines/releases/download/0.9.0/kabanero-events-pipelines.tar.gz ``` -When the product operator activates the Kabanero CRD, it associates the pipelines in the pipelines archive with each of the stacks in the stack hub. The default pipelines are intended to work with all the stacks in the stack hub in the previous example. When the operator activates all the pipeline resources (such as the tasks, trigger bindings, and pipelines) in the archive, it will add a suffix to the name of the resource with the shorted digest of the pipelines archive. This provides an easy way to have multiple versions of the same pipeline active on the cluster. +When the product operator activates the Kabanero CRD, it associates the pipelines in the pipelines archive with each of the stacks in the stack hub. The default pipelines are intended to work with all the stacks in the stack hub in the previous example. When the operator activates all the pipeline resources (such as the tasks, trigger bindings, and pipelines) in the archive, it will add a suffix to the name of the resource with the shorted digest of the pipelines archive. This provides an easy way to have multiple versions of the same pipeline to be active on the cluster. ### Incubator pipelines @@ -179,17 +223,17 @@ metadata: name: kabanero namespace: kabanero spec: - version: "0.6.0" + version: "0.9.0" stacks: repositories: - name: central https: - url: https://github.com/kabanero-io/stacks/releases/download/0.6.0/kabanero-index.yaml + url: https://github.com/kabanero-io/stacks/releases/download/0.9.0/kabanero-index.yaml pipelines: - id: default sha256: 14d59b7ebae113c18fb815c2ccfd8a846c5fbf91d926ae92e0017ca5caf67c95 https: - url: https://github.com/kabanero-io/kabanero-pipelines/releases/download/0.6.0/default-kabanero-pipelines.tar.gz + url: https://github.com/kabanero-io/kabanero-pipelines/releases/download/0.9.0/default-kabanero-pipelines.tar.gz ``` When the product operator activates the CRD, it associates the pipelines in the pipelines archive with each of the stacks in the stack hub. The default pipelines are intended to work with all the stacks in the stack hub in the previous example. All of the pipeline-related resources (such as the tasks, trigger bindings, and pipelines) prefix the name of the resource with the keyword `StackId`. When the operator activates these resources, it replaces the keyword with the name of the stack it is activating. From 0ccd0ffca1de813282818d9bea8f3675516ebccb Mon Sep 17 00:00:00 2001 From: Jeff Antley Date: Fri, 15 May 2020 15:23:26 -0400 Subject: [PATCH 09/10] Update working-with-pipelines.md --- .../working-with-pipelines.md | 47 +++++++++---------- 1 file changed, 22 insertions(+), 25 deletions(-) diff --git a/publish/working-with-pipelines/working-with-pipelines.md b/publish/working-with-pipelines/working-with-pipelines.md index 2d57dc1..41f9c4f 100644 --- a/publish/working-with-pipelines/working-with-pipelines.md +++ b/publish/working-with-pipelines/working-with-pipelines.md @@ -55,7 +55,7 @@ A set of Kabanero tasks and pipelines are provided in the [Kabanero pipelines re ### Events pipelines -The tasks and pipelines provided in the [Kabanero pipelines repository events directory](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/incubator/events) are geared to work with the Kabanero events operator. Follow the instructions in to set up the organization webhook and EventMediator to drive these pipelines. +The tasks and pipelines provided in the [Kabanero pipelines repository events directory](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/incubator/events) are geared to work with the Kabanero events operator. Follow the instructions in the [Integrating the events operator](../integrating-events-operator/integating-events-operator.html) guide to set up the organization webhook and EventMediator to drive these pipelines. There are four primary pipelines provided here to help illustrate the following work flow. @@ -69,7 +69,7 @@ This action triggers the [build-push-promote-pl.yaml](https://github.com/kabaner The pipeline invokes the following tasks to accomplish the steps listed: * [build-push-promote-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/build-push-promote-task.yaml) - This task first does a pre-build goverance policy check to validate the stack version in the application repository is allowed to build based on the goverance policy that is configured. It then builds a container image from the artifacts in the git-source repository by using `appsody build`. The appsody build command leverages [Buildah](https://github.com/containers/buildah) to build the image. The command also generates the `app-deploy.yaml` that is used for deployment. If there is already a copy of the `app-deploy.yaml` file in the source repository, it is merged with the new one generated by this step. After the image is built, the image is then optionally signed if the necessary configuration is setup. Refer to the [image signing operator](https://github.com/kabanero-io/kabanero-security/tree/master/pipelines/samples/signing-operator) for more information on configuring image signing. The image is then pushed to the configured image registry. + This task first does a pre-build governance policy check to validate the stack version in the application repository is allowed to build based on the governance policy that is configured. It then builds a container image from the artifacts in the git-source repository by using `appsody build`. The appsody build command leverages [Buildah](https://github.com/containers/buildah) to build the image. The command also generates the `app-deploy.yaml` that is used for deployment. If there is already a copy of the `app-deploy.yaml` file in the source repository, it is merged with the new one generated by this step. After the image is built, the image is then optionally signed if the necessary configuration is setup. Refer to the [image signing operator](https://github.com/kabanero-io/kabanero-security/tree/master/pipelines/samples/signing-operator) for more information on configuring image signing. The image is then pushed to the configured image registry. (Tech preview feature) A configmap called `gitops-map` in the Kabanero namespace can optionally be configured to promote the service to a GitOps repo after the build. The step will invoke the [`services promote`](https://github.com/rhd-gitops-example/services) command to create a PR with the updated `app-deploy.yaml` file in the configured GitOps repo. The following key value pairs should be setup in the configmap: ``` @@ -101,17 +101,17 @@ The pipeline invokes the following tasks to accomplish the steps listed: ``` * [deploy-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-task.yaml) - If the `webhooks-tekton-local-deploy` property is set to `true` in the mediator, the image is deployed to the namespace configured in the `app-deploy.yaml`. By default, the application will be deployed in the `kabanero` namespace. + If the `webhooks-tekton-local-deploy` property is set to `true` in the mediator, the image is deployed to the namespace configured in the `app-deploy.yaml`. By default, the application will be deployed in the `kabanero` namespace. * [image-scan-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/image-scan-task.yaml) - The `image-scan-task` task will initiate a container scan of the image published by the `build-push-task` using OpenSCAP. The results of the scan are published in the logs of the task. + The `image-scan-task` task initiates a container scan of the image published by the `build-push-task` using OpenSCAP. The results of the scan are published in the logs of the task. * **A release of the application is created** -This will trigger the [image-retag-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/image-retag-pl.yaml) pipeline which leverages the [image-retag-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/image-retag-task.yaml) to create a new tag of the image to match with the git release. +This event triggers the [image-retag-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/image-retag-pl.yaml) pipeline, which leverages the [image-retag-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/image-retag-task.yaml) to create a new tag of the image to match with the git release. -* **The pull request in the gitops repo is merged** (This feature is Tech Preview in this release) -When the PR that was created by the promote step of the `build-push-promote-pl` is merged in the gitops repo, it will trigger the [deploy-kustomize-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-kustomize-pl.yaml) pipeline, which leverages the [deploy-kustomize-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-kustomize-task.yaml) to trigger a deployment to the environment configured in the gitops repo. +* **The pull request in the GitOps repo is merged** (This feature is Tech Preview in this release) +When the PR that was created by the promote step of the `build-push-promote-pl` is merged in the GitOps repository, it triggers the [deploy-kustomize-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-kustomize-pl.yaml) pipeline, which leverages the [deploy-kustomize-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-kustomize-task.yaml) to trigger a deployment to the environment configured in the GitOps repository. ### Incubator pipelines @@ -121,7 +121,7 @@ Details of some of the primary pipelines and tasks: - [build-deploy-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/build-deploy-pl.yaml) -This is the primary pipeline that showcases a majority of the tasks supplied in the Kabanero pipelines incubator repo. It enforces governance policy, builds the code, optionally signs the image, pushes it to the image registry, scans the image, and can conditionally deploy the image on the cluster. When running the pipeline via a webhook, the pipeline leverages the triggers functionality to conditionally deploy the application only when a pull request is merged in the git repo. Other actions that trigger the pipeline run enforce governance policy, build, push, and scan the image. +This is the primary pipeline that showcases a majority of the tasks supplied in the Kabanero pipelines incubator repository. It enforces governance policy, builds the code, optionally signs the image, pushes it to the image registry, scans the image, and can conditionally deploy the image on the cluster. When running the pipeline via a webhook, the pipeline leverages the triggers functionality to conditionally deploy the application only when a pull request is merged in the GitHub repository. Other actions that trigger the pipeline run enforce governance policy, build, push, and scan the image. - [build-push-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/build-push-task.yaml) @@ -147,9 +147,9 @@ The [build-push-promote-pl.yaml](https://github.com/kabanero-io/kabanero-pipelin The pipeline invokes the following tasks to accomplish the steps listed: * [build-push-promote-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/experimental/gitops/build-push-promote-task.yaml) - This task first does a pre-build goverance policy check to validate the stack version in the application repository is allowed to build based on the goverance policy that is configured. It then builds a container image from the artifacts in the git-source repository by using `appsody build`. The appsody build command leverages [Buildah](https://github.com/containers/buildah) to build the image. The image is then optionally signed and pushed to the configured image registry. + This task first does a pre-build governance policy check to validate the stack version in the application repository is allowed to build based on the governance policy that is configured. It then builds a container image from the artifacts in the git-source repository by using `appsody build`. The appsody build command leverages [Buildah](https://github.com/containers/buildah) to build the image. The image is then optionally signed and pushed to the configured image registry. - A configmap called `gitops-map` in the Kabanero namespace can optionally be configured to promote the service to a GitOps repo after the build. The step will invoke the [`services promote`](https://github.com/rhd-gitops-example/services) command to create a PR with the updated `app-deploy.yaml` file in the configured GitOps repo. The following key value pairs should be setup in the configmap: + A `configmap` file called `gitops-map` in the Kabanero namespace can optionally be configured to promote the service to a GitOps repository after the build. The step will invoke the [`services promote`](https://github.com/rhd-gitops-example/services) command to create a PR with the updated `app-deploy.yaml` file in the configured GitOps repository. The following key value pairs should be set up in the `configmap` file: ``` kind: ConfigMap apiVersion: v1 @@ -163,7 +163,7 @@ The pipeline invokes the following tasks to accomplish the steps listed: gitops-commit-user-email: ``` - A secret called `gitops-token` also has to be created in the Kabanero namspace. This example yaml creates the secret. + A secret called `gitops-token` also has to be created in the Kabanero namspace. This example yaml creates the secret: ``` apiVersion: v1 kind: Secret @@ -181,7 +181,7 @@ The pipeline invokes the following tasks to accomplish the steps listed: The `image-scan-task` task initiates a container scan of the image published by the `build-push-task` using OpenSCAP. The results of the scan are published in the logs of the task. -The [deploy-kustomize-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/experimental/gitops/deploy-kustomize-pl.yaml) pipeline, which leverages the [deploy-kustomize-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/experimental/gitops/deploy-kustomize-task.yaml) to trigger a deployment to the environment configured in the GitOps repo. +The [deploy-kustomize-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/experimental/gitops/deploy-kustomize-pl.yaml) pipeline, which leverages the [deploy-kustomize-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/experimental/gitops/deploy-kustomize-task.yaml) to trigger a deployment to the environment configured in the GitOps repository. # Before running pipelines @@ -210,7 +210,7 @@ spec: url: https://github.com/kabanero-io/kabanero-pipelines/releases/download/0.9.0/kabanero-events-pipelines.tar.gz ``` -When the product operator activates the Kabanero CRD, it associates the pipelines in the pipelines archive with each of the stacks in the stack hub. The default pipelines are intended to work with all the stacks in the stack hub in the previous example. When the operator activates all the pipeline resources (such as the tasks, trigger bindings, and pipelines) in the archive, it will add a suffix to the name of the resource with the shorted digest of the pipelines archive. This provides an easy way to have multiple versions of the same pipeline to be active on the cluster. +When the product operator activates the Kabanero CRD, it associates the pipelines in the pipelines archive with each of the stacks in the stack hub. The default pipelines are intended to work with all the stacks in the stack hub in the previous example. When the operator activates all the pipeline resources (such as the tasks, trigger bindings, and pipelines) in the archive, it adds a suffix to the name of the resource with the shorted digest of the pipelines archive. This configuration provides an easy way to have multiple versions of the same pipeline to be active on the cluster. ### Incubator pipelines @@ -240,7 +240,7 @@ When the product operator activates the CRD, it associates the pipelines in the ## Creating and updating your own tasks and pipelines -The default tasks and pipelines can be updated by forking the Kabanero Pipelines repo and editing the files under `pipelines/`. The easiest way to generate the archive for use by the Kabanero CRD is to run the [package.sh](https://github.com/kabanero-io/kabanero-pipelines/blob/master/ci/package.sh) script from the root directory of the pipelines project. The script generates the archive files with the necessary pipeline artifacts and a `manifest.yaml` file that describes the contents of the archive. It generates the pipelines archive file under `ci/assests`. It generates seperate archives for the legacy incubator pipelines, events pipelines, and the experimental gitops pipleines. +The default tasks and pipelines can be updated by forking the Kabanero Pipelines repo and editing the files under `pipelines/`. An easy way to generate the archive for use by the Kabanero CRD is to run the [package.sh](https://github.com/kabanero-io/kabanero-pipelines/blob/master/ci/package.sh) script from the root directory of the pipelines project. The script generates the archive files with the necessary pipeline artifacts and a `manifest.yaml` file that describes the contents of the archive. It generates the pipelines archive file under `ci/assests`. It generates separate archives for the legacy incubator pipelines, events pipelines, and the experimental GitOps pipleines. Alternatively, you can run the Travis build against a release of your pipelines repo, which also generates the archive file with a `manifest.yaml` file and attaches it to your release. @@ -388,11 +388,11 @@ Follow these steps: ## Running pipelines by using the pipelines dashboard webhook extension (Deprecated) -You can use the [pipelines dashboard webhook extension](https://github.com/tektoncd/experimental/blob/master/webhooks-extension/docs/GettingStarted.md) to drive pipelines that automatically build and deploy an application whenever you update the code in your GitHub repo. Events such as commits or pull requests can be set up to automatically trigger pipeline runs. +You can use the [pipelines dashboard webhook extension](https://github.com/tektoncd/experimental/blob/master/webhooks-extension/docs/GettingStarted.md) to drive pipelines that automatically build and deploy an application whenever you update the code in your GitHub repository. Events such as commits or pull requests can be set up to automatically trigger pipeline runs. ## Running pipelines by using a script -If you are developing a new pipeline and want to test it in a tight loop, you might want to use a script or manually drive the pipeline. +If you are developing a new pipeline and want to test it in a tight loop, you can use a script or manually drive the pipeline. 1. Log in to your cluster. For example, @@ -443,7 +443,7 @@ Follow these steps to run a pipeline directly from the command line: 1. Create pipeline resources. - Use the `pipeline-resource-template.yaml` file to create the `PipelineResources`. The `pipeline-resource-template.yaml` is provided in the pipelines [/pipelines/sample-helper-files](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/sample-helper-files) directory. Update the docker-image URL. You can use the sample GitHub repo or update it to point to your own GitHub repo. + Use the `pipeline-resource-template.yaml` file to create the `PipelineResources`. The `pipeline-resource-template.yaml` is provided in the pipelines [/pipelines/sample-helper-files](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/sample-helper-files) directory. Update the docker-image URL. You can use the sample GitHub repository or update it to point to your own GitHub repository. 1. After you update the file, apply it as shown in the following example: @@ -453,7 +453,7 @@ Follow these steps to run a pipeline directly from the command line: ## Activating tasks and pipelines -The installations that activate the featured application stacks also activate the tasks and pipelines. If you are creating a new task or pipeline, activate it manually, as shown in the following example. +The installations that activate the featured application stacks also activate the tasks and pipelines. If you are creating a new task or pipeline, activate it manually, as shown in this example: ```shell oc apply -f @@ -462,7 +462,7 @@ oc apply -f ## Running the pipeline -A sample `manual-pipeline-run-template.yaml` file is provided in the [/pipelines/sample-helper-files](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/sample-helper-files) directory. Rename the template file to a name of your choice (for example, pipeline-run.yaml), and update the file to replace `application-stack-name` with the name of your application stack. After you update the file, run it as shown in the following example. +A sample `manual-pipeline-run-template.yaml` file is provided in the [/pipelines/sample-helper-files](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/sample-helper-files) directory. Rename the template file to a name of your choice (for example, pipeline-run.yaml), and update the file to replace `application-stack-name` with the name of your application stack. After you update the file, run it as shown in this example: ```shell oc apply -f -pipeline-run.yaml @@ -476,16 +476,13 @@ oc apply -f -pipeline-run.yaml ## Checking the status of the pipeline run -You can check the status of the pipeline run from the Kubernetes console, -command line, or pipelines dashboard. +You can check the status of the pipeline run from the Kubernetes console, command line, or pipelines dashboard. ### Checking pipeline run status from the pipelines dashboard -1. Log in to the pipelines dashboard and click `Pipeline runs' -in the sidebar menu. +1. Log in to the pipelines dashboard and click `Pipeline runs' in the sidebar menu. -1. Find your pipeline run in the list and click it to check the status and find logs. You can see logs -and status for each step and task. +1. Find your pipeline run in the list and click it to check the status and find logs. You can see logs and status for each step and task. ### Checking pipeline run status from the command line From 12227038fd7a78519dcf508f6315292e0a9d4174 Mon Sep 17 00:00:00 2001 From: Jeff Antley Date: Fri, 15 May 2020 15:35:17 -0400 Subject: [PATCH 10/10] Update working-with-pipelines.md --- .../working-with-pipelines.md | 30 +++++++++---------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/publish/working-with-pipelines/working-with-pipelines.md b/publish/working-with-pipelines/working-with-pipelines.md index 41f9c4f..ad63d30 100644 --- a/publish/working-with-pipelines/working-with-pipelines.md +++ b/publish/working-with-pipelines/working-with-pipelines.md @@ -42,7 +42,7 @@ guide-category: pipelines - sign the image - retag an image - deploy the application to the Kubernetes cluster -- promote a service to a GitOps repo (This feature is Tech Preview in this release) +- promote a service to a GitOps repository (This feature is Tech Preview in this release) - deploy a complete microservice scenario using Kustomize (This feature is Tech Preview in this release) You can also create your own tasks and pipelines and customize the pre-built pipelines and tasks. All tasks and pipelines are activated by the product operator. @@ -71,7 +71,7 @@ The pipeline invokes the following tasks to accomplish the steps listed: * [build-push-promote-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/build-push-promote-task.yaml) This task first does a pre-build governance policy check to validate the stack version in the application repository is allowed to build based on the governance policy that is configured. It then builds a container image from the artifacts in the git-source repository by using `appsody build`. The appsody build command leverages [Buildah](https://github.com/containers/buildah) to build the image. The command also generates the `app-deploy.yaml` that is used for deployment. If there is already a copy of the `app-deploy.yaml` file in the source repository, it is merged with the new one generated by this step. After the image is built, the image is then optionally signed if the necessary configuration is setup. Refer to the [image signing operator](https://github.com/kabanero-io/kabanero-security/tree/master/pipelines/samples/signing-operator) for more information on configuring image signing. The image is then pushed to the configured image registry. - (Tech preview feature) A configmap called `gitops-map` in the Kabanero namespace can optionally be configured to promote the service to a GitOps repo after the build. The step will invoke the [`services promote`](https://github.com/rhd-gitops-example/services) command to create a PR with the updated `app-deploy.yaml` file in the configured GitOps repo. The following key value pairs should be setup in the configmap: + (Tech preview feature) A configmap called `gitops-map` in the Kabanero namespace can optionally be configured to promote the service to a GitOps repository after the build. The step will invoke the [`services promote`](https://github.com/rhd-gitops-example/services) command to create a PR with the updated `app-deploy.yaml` file in the configured GitOps repository. The following key value pairs should be setup in the configmap: ``` kind: ConfigMap apiVersion: v1 @@ -110,7 +110,7 @@ The pipeline invokes the following tasks to accomplish the steps listed: This event triggers the [image-retag-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/image-retag-pl.yaml) pipeline, which leverages the [image-retag-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/image-retag-task.yaml) to create a new tag of the image to match with the git release. -* **The pull request in the GitOps repo is merged** (This feature is Tech Preview in this release) +* **The pull request in the GitOps repository is merged** (This feature is Tech Preview in this release) When the PR that was created by the promote step of the `build-push-promote-pl` is merged in the GitOps repository, it triggers the [deploy-kustomize-pl.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-kustomize-pl.yaml) pipeline, which leverages the [deploy-kustomize-task.yaml](https://github.com/kabanero-io/kabanero-pipelines/blob/master/pipelines/incubator/events/deploy-kustomize-task.yaml) to trigger a deployment to the environment configured in the GitOps repository. ### Incubator pipelines @@ -137,7 +137,7 @@ This is the primary pipeline that showcases a majority of the tasks supplied in The `image-scan-task` task initiates a container scan of the image published by the `build-push-task` using OpenSCAP. The results of the scan are published in the logs of the task. -For more tasks and pipelines, see [the kabanero-pipelines repo](https://github.com/kabanero-io/kabanero-pipelines). +For more tasks and pipelines, see [the kabanero-pipelines repository](https://github.com/kabanero-io/kabanero-pipelines). ### Experimental GitOps pipelines (Tech Preview) @@ -240,9 +240,9 @@ When the product operator activates the CRD, it associates the pipelines in the ## Creating and updating your own tasks and pipelines -The default tasks and pipelines can be updated by forking the Kabanero Pipelines repo and editing the files under `pipelines/`. An easy way to generate the archive for use by the Kabanero CRD is to run the [package.sh](https://github.com/kabanero-io/kabanero-pipelines/blob/master/ci/package.sh) script from the root directory of the pipelines project. The script generates the archive files with the necessary pipeline artifacts and a `manifest.yaml` file that describes the contents of the archive. It generates the pipelines archive file under `ci/assests`. It generates separate archives for the legacy incubator pipelines, events pipelines, and the experimental GitOps pipleines. +The default tasks and pipelines can be updated by forking the Kabanero Pipelines repository and editing the files under `pipelines/`. An easy way to generate the archive for use by the Kabanero CRD is to run the [package.sh](https://github.com/kabanero-io/kabanero-pipelines/blob/master/ci/package.sh) script from the root directory of the pipelines project. The script generates the archive files with the necessary pipeline artifacts and a `manifest.yaml` file that describes the contents of the archive. It generates the pipelines archive file under `ci/assests`. It generates separate archives for the legacy incubator pipelines, events pipelines, and the experimental GitOps pipleines. -Alternatively, you can run the Travis build against a release of your pipelines repo, which also generates the archive file with a `manifest.yaml` file and attaches it to your release. +Alternatively, you can run the Travis build against a release of your pipelines repository, which also generates the archive file with a `manifest.yaml` file and attaches it to your release. For more detailed instructions, see [Curating Pipelines](../curating-pipelines/curating-pipelines.html) @@ -360,9 +360,9 @@ Explore how to use pipelines to build and manage application stacks. 1. [Kabanero foundation](https://github.com/kabanero-io/kabanero-foundation) must be installed on a supported Kubernetes deployment. -1. A persistent volume must be configured. See the following section for details. +1. A persistent volume must be configured. See the **Getting started** section for details. -1. Secrets for the git repo (if private) and image repository +1. Secrets for the GitHub repository (if private) and image repository ## Getting started @@ -400,25 +400,25 @@ If you are developing a new pipeline and want to test it in a tight loop, you ca oc login :8443 ``` -1. Clone the pipelines repo +1. Clone the pipelines repository: ```shell git clone https://github.com/kabanero-io/kabanero-pipelines ``` -1. Run the following script with the appropriate parameters +1. Run this script with the appropriate parameters: ```shell cd ./pipelines/sample-helper-files/./manual-pipeline-run-script.sh -r [git_repo of the Appsody project] -i [docker registery path of the image to be created] -c [application stack name of which pipeline to be run]" ``` - - The following example is configured to use the dockerhub container registry: + - When using the dockerhub container registry: ```shell ./manual-pipeline-run-script.sh -r https://github.com/mygitid/appsody-test-project -i index.docker.io/mydockeid/my-java-openliberty-image -c java-openliberty" ``` - - The following example is configured to use the local OpenShift container registry: + - When using the local OpenShift container registry: ```shell ./manual-pipeline-run-script.sh -r https://github.com/mygitid/appsody-test-project -i docker-registry.default.svc:5000/kabanero/my-java-openliberty-image -c java-openliberty" @@ -445,7 +445,7 @@ Follow these steps to run a pipeline directly from the command line: Use the `pipeline-resource-template.yaml` file to create the `PipelineResources`. The `pipeline-resource-template.yaml` is provided in the pipelines [/pipelines/sample-helper-files](https://github.com/kabanero-io/kabanero-pipelines/tree/master/pipelines/sample-helper-files) directory. Update the docker-image URL. You can use the sample GitHub repository or update it to point to your own GitHub repository. -1. After you update the file, apply it as shown in the following example: +1. After you update the file, apply it as shown in this example: ```shell oc apply -f -pipeline-resources.yaml @@ -480,7 +480,7 @@ You can check the status of the pipeline run from the Kubernetes console, comman ### Checking pipeline run status from the pipelines dashboard -1. Log in to the pipelines dashboard and click `Pipeline runs' in the sidebar menu. +1. Log in to the pipelines dashboard and click `Pipeline runs` in the sidebar menu. 1. Find your pipeline run in the list and click it to check the status and find logs. You can see logs and status for each step and task. @@ -495,7 +495,7 @@ oc -n kabanero describe pipelinerun.tekton.dev/ You can also see pods for the pipeline runs, for which you can specify `oc describe` and `oc logs` to get more details. -If the pipeline run was successful, you can see a Docker image in our Docker registry and a pod that’s running your application. +If the pipeline run was successful, you can see a Docker image in our Docker registry and a pod that is running your application.