Merge pull request #56 from MoOyeg/main

Added Feature for Image and Pipeline Signing

README.md

@@ -11,11 +11,11 @@ DevSecOps CICD pipeline demo using several technologies such as:
 
 Vulnerability and configuration management methods included in this demo are the following:
 
-* **Static application security testing (SAST)**, which analyzes code under development for vulnerabilities and quality issues.
-* **Software composition analysis (SCA)**, which examines dependent packages included with applications, looking for known vulnerabilities and licensing issues.
-* **Interactive application security testing (IAST)** and **dynamic application security testing (DAST)** tools, which analyze running applications to find execution vulnerabilities.
-* **Configuration management** with analysis and management of application and infrastructure configurations in DevOps. Traditionally this was not used as a way to improve security. But properly managing configurations in a GitOps process can strengthen security by improving change controls, identifying configuration defects that can reduce the attack surface, and signing and tracking authorship for better accountability and opportunities to improve.
-* **Image risk** is any risk associated with a container image. This includes vulnerable dependencies, embedded secrets, bad configurations, malware, or images that are not trusted.
+- **Static application security testing (SAST)**, which analyzes code under development for vulnerabilities and quality issues.
+- **Software composition analysis (SCA)**, which examines dependent packages included with applications, looking for known vulnerabilities and licensing issues.
+- **Interactive application security testing (IAST)** and **dynamic application security testing (DAST)** tools, which analyze running applications to find execution vulnerabilities.
+- **Configuration management** with analysis and management of application and infrastructure configurations in DevOps. Traditionally this was not used as a way to improve security. But properly managing configurations in a GitOps process can strengthen security by improving change controls, identifying configuration defects that can reduce the attack surface, and signing and tracking authorship for better accountability and opportunities to improve.
+- **Image risk** is any risk associated with a container image. This includes vulnerable dependencies, embedded secrets, bad configurations, malware, or images that are not trusted.
 
 This pipeline also improve security adding the following Open Source components:
 
@@ -95,6 +95,38 @@ These policies notification can be enabled by each system policy enabled in our
 
 NOTE: By now the integration is manual. WIP to automate it.
 
+## 6. Image Signing and Pipeline Signing
+
+The original demo can be extended to use Cosign to Sign Image artifacts and also to sign the Tekton Build Pipeline via Tekton [Chaining](https://github.com/tektoncd/chains).
+
+To extend the pipeline run the extend.sh script
+
+```sh
+   ./extend.sh
+```
+
+This will install Noobaa(Object Storage), Quay, and create a pod for cosign secret generation and verification.It will also install the tekton chains operator and integrate with ACS policies to generate violations for non signed images.
+
+After installation the pipeline will build images to quay and have a task that signs the image.  
+<img align="center" width="750" src="docs/pics/pipeline-with-sign-task.png">
+
+We also create a policy in ACS that will generate a violation for every unsigned image
+<img align="center" width="750" src="docs/pics/acs-trusted-signature-violation.png">
+
+Pipeline can be run normally via the Run the demo Instructions below.
+
+After Pipeline is run Quay will show the image signed by Cosign
+<img align="center" width="750" src="docs/pics/quay-with-signatures.png">
+
+Since we have Tekton Chaining enabled, successfully completed Taskruns will also be annotated with cosign signatures and payload information.
+<img align="center" width="750" src="docs/pics/taskrun.png">
+
+And we can verify the signature and payload information of our last successful pipelinerun using the below command.
+
+```sh
+   ./demo.sh sign-verify
+```
+
 ## Security Policies and CI Violations
 
 In this demo, we can control the security policies applied into our pipelines, scanning the images and analysing the different deployments templates used for deploy our applications.
@@ -114,9 +146,10 @@ This ensures that we have the total control of our pipelines, and no image is pu
 To show a complete demo and show the transition from a "bad image" to an image that passes the build enforcement, we can update the Tekton task of the image build and fix the image. In this example, we will be enabling the enforcement of the "Red Hat Package Manager in Image" policy in ACS, which will fail our pipeline at the image-check as both `yum` and `rpm` package managers are present in our base image.
 
 Update the tekton task:
+
 1. Delete the `s2i-java-11` task
-    1. With the UI: From the OpenShift UI, make sure you are in the cicd project and then go to Pipelines > Tasks and delete the `s2i-java-11` task.
-    2. With the Tekton cli `tkn task delete s2i-java-11`
+   1. With the UI: From the OpenShift UI, make sure you are in the cicd project and then go to Pipelines > Tasks and delete the `s2i-java-11` task.
+   2. With the Tekton cli `tkn task delete s2i-java-11`
 2. Apply the new update task: `kubectl apply -f fix-image/s2ijava-mgr.yaml`
 3. Re-run the pipeline, your deployment now succeeds.
 
@@ -188,20 +221,20 @@ NOTE: This pipeline will fail if you don't [disable the "Fixable at least Import
 
 ## Quick Video with the Demo
 
-* [Option I - Complete CICD End2End process (Success)](https://youtu.be/uA7nUYchY5Q)
+- [Option I - Complete CICD End2End process (Success)](https://youtu.be/uA7nUYchY5Q)
 
-* [Option II - Failure CICD pipeline due to the ACS violation policy](https://youtu.be/jTRImofd6wQ?t=380)
+- [Option II - Failure CICD pipeline due to the ACS violation policy](https://youtu.be/jTRImofd6wQ?t=380)
 
-* [Openshift Coffee Break - ACS for Kubernetes - DevSecOps Way](https://youtu.be/43Mr30mXq0I?t=1955)
+- [Openshift Coffee Break - ACS for Kubernetes - DevSecOps Way](https://youtu.be/43Mr30mXq0I?t=1955)
 
 ## Promote Pipeline and Triggers
 
-* [Promote Pipeline](docs/promote.md)
-* [Triggers in Dev Pipeline](doc/triggers.md)
+- [Promote Pipeline](docs/promote.md)
+- [Triggers in Dev Pipeline](doc/triggers.md)
 
 # Troubleshooting
 
-* [Check the Tshoot section](docs/tshoot.md)
+- [Check the Tshoot section](docs/tshoot.md)
 
 # Credits
 

bootstrap/deploy_demo.yaml

@@ -28,3 +28,4 @@
     - name: 'Install the ACS Post Content'
       include_role:
         name: "ocp4-post-acs"
+

bootstrap/deploy_signing.yaml

@@ -0,0 +1,18 @@
+---
+- name: 'Extend Original Demo for Image and TaskRun Signing'
+  hosts: localhost
+  connection: local
+  tasks:
+    - name: 'Install NooBaa'
+      include_role:
+        name: "ocp4-install-noobaa"
+
+    - name: 'Install and configure Quay'
+      include_role:
+        name: "ocp4-install-quay"
+
+    - name: 'Install and Enable the infra for Signing and Tekton Chaining'
+      include_role:
+        name: "ocp4-install-signing"
+
+

bootstrap/roles/ocp4-install-noobaa/defaults/main.yaml

@@ -0,0 +1,3 @@
+noobaa_storage_class: ""
+noobaa_size: "50Gi"
+backing_store_name: "noobaa-pv-backing-store"

bootstrap/roles/ocp4-install-noobaa/tasks/main.yaml

@@ -0,0 +1,2 @@
+
+- import_tasks: noobaa-create.yaml

bootstrap/roles/ocp4-install-noobaa/tasks/noobaa-create.yaml

@@ -0,0 +1,125 @@
+- name: Get cluster version
+  k8s_info:
+    api_version: config.openshift.io/v1
+    kind: ClusterVersion
+    name: version
+  register: r_cluster_version
+
+- name: Set ocp4_cluster_version fact
+  set_fact:
+    ocp4_cluster_version: "{{ r_cluster_version.resources[0].status.history[0].version }}"
+
+- name: Obtain Channel from Version
+  set_fact:
+    ocp4_channel: "{{ ocp4_cluster_version.split('.') }}"
+
+- name: Set Openshift Channel
+  set_fact:
+    ocp4_channel: "stable-{{ ocp4_channel[0] + '.' + ocp4_channel[1] }}"
+
+- name: Print OpenShift version
+  debug:
+    msg: "{{ ocp4_channel }}"
+
+- name: Adapt to the openshift_cluster_version LESS than 4.9
+  when: ocp4_cluster_version is version_compare('4.9', '<')
+  block:
+    - name: Create OpenShift Objects to install Noobaa
+      k8s:
+        state: present
+        definition: "{{ lookup('template', item ) | from_yaml }}"
+      loop:
+      - ./templates/odf-namespace.yaml.j2
+      - ./templates/operatorgroup-storage.yaml.j2
+      - ./templates/ocs-subscription.yaml.j2
+
+    - name: Wait for NooBaa CRD to exist
+      kubernetes.core.k8s_info:
+        api_version: "apiextensions.k8s.io/v1beta1"
+        kind: CustomResourceDefinition
+        name: "noobaas.noobaa.io"
+      register: crds
+      until: crds.resources|length > 0
+      retries: 30
+      delay: 10
+
+- name: Adapt to the openshift_cluster_version MORE than 4.9
+  when: ocp4_cluster_version is version_compare('4.9', '>=')
+  block:
+    - name: Create OpenShift Objects to install Noobaa
+      k8s:
+        state: present
+        definition: "{{ lookup('template', item ) | from_yaml }}"
+      loop:
+      - ./templates/odf-namespace.yaml.j2
+      - ./templates/operatorgroup-storage.yaml.j2
+      - ./templates/odf-subscription.yaml.j2
+
+    - name: Wait for NooBaa CRD to exist
+      kubernetes.core.k8s_info:
+        api_version: "apiextensions.k8s.io/v1"
+        kind: CustomResourceDefinition
+        name: "noobaas.noobaa.io"
+      register: crds
+      until: crds.resources|length > 0
+      retries: 30
+      delay: 10
+
+- name: Create Noobaa Object
+  k8s:
+    state: present
+    definition: "{{ lookup('template', item ) | from_yaml }}"
+  loop:
+  - ./templates/noobaa-object.yaml.j2
+
+- name: Wait Until NooBaa Object is Ready
+  shell: |
+    oc get noobaas.noobaa.io/noobaa -n openshift-storage -o jsonpath='{.status.phase}'
+  register: noobaa_status
+  retries: 10
+  delay: 20
+  until:
+    - noobaa_status.stdout == "Ready"
+
+- name: Get Default Openshift Storage Class
+  shell: |
+    oc get sc -o=jsonpath='{.items[?(@.metadata.annotations.storageclass\.kubernetes\.io/is-default-class=="true")].metadata.name}'
+  register: default_openshift_storage_class
+  when: noobaa_storage_class == ""
+
+- name: Get any other Storage Class
+  shell: |
+    oc get sc -o name | head -n 1 | cut -d "/" -f2
+  register: other_openshift_storage_class
+  when: (default_openshift_storage_class.stdout |default("") == "" ) and (noobaa_storage_class == "")
+
+- name: Use default storage class if it was set
+  ansible.builtin.set_fact:
+    noobaa_storage_class: "{{ default_openshift_storage_class.stdout }}"
+  when: (default_openshift_storage_class.stdout |default("") != "" ) and (noobaa_storage_class == "")
+
+- name: Try other possible storage class if no defined/default storage class
+  ansible.builtin.set_fact:
+    noobaa_storage_class: "{{ other_openshift_storage_class.stdout }}"
+  when: (default_openshift_storage_class.stdout |default("") == "" ) and (noobaa_storage_class == "") and (other_openshift_storage_class|default("") != "")
+
+- name: Create NooBaa Backing Store
+  k8s:
+    state: present
+    definition: "{{ lookup('template', item ) | from_yaml }}"
+  loop:
+  - ./templates/noobaa-backingstore.yaml.j2
+
+- name: Wait Until NooBaa Object is Ready
+  shell: |
+    oc get BackingStore/"{{ backing_store_name }}" -n openshift-storage -o jsonpath='{.status.phase}'
+  register: backing_store
+  retries: 10
+  delay: 20
+  until:
+    - backing_store.stdout == "Ready"
+
+- name: Patch Bucket Class with Backing Store
+  shell: |
+    oc patch bucketclass noobaa-default-bucket-class --patch '{"spec":{"placementPolicy":{"tiers":[{"backingStores":["{{backing_store_name}}"]}]}}}' --type merge -n openshift-storage
+  

bootstrap/roles/ocp4-install-noobaa/templates/noobaa-backingstore.yaml.j2

@@ -0,0 +1,17 @@
+apiVersion: noobaa.io/v1alpha1
+kind: BackingStore
+metadata:
+  finalizers:
+  - noobaa.io/finalizer
+  labels:
+    app: noobaa
+  name: noobaa-pv-backing-store
+  namespace: openshift-storage
+spec:
+  pvPool:
+    numVolumes: 1
+    resources:
+      requests:
+        storage: {{ noobaa_size }}
+    storageClass: {{ noobaa_storage_class }}
+  type: pv-pool

bootstrap/roles/ocp4-install-noobaa/templates/noobaa-object.yaml.j2

@@ -0,0 +1,15 @@
+apiVersion: noobaa.io/v1alpha1
+kind: NooBaa
+metadata:
+  name: noobaa
+  namespace: openshift-storage
+spec:
+ dbResources:
+   requests:
+     cpu: '0.1'
+     memory: 1Gi
+ dbType: postgres
+ coreResources:
+   requests:
+     cpu: '0.1'
+     memory: 1Gi

bootstrap/roles/ocp4-install-noobaa/templates/ocs-subscription.yaml.j2

@@ -0,0 +1,11 @@
+apiVersion: operators.coreos.com/v1alpha1
+kind: Subscription
+metadata:
+  name: ocs-operator
+  namespace: openshift-storage
+spec:
+  channel: {{ ocp4_channel }}
+  installPlanApproval: Automatic
+  name: ocs-operator
+  source: redhat-operators
+  sourceNamespace: openshift-marketplace

bootstrap/roles/ocp4-install-noobaa/templates/odf-namespace.yaml.j2

@@ -0,0 +1,8 @@
+kind: Namespace
+apiVersion: v1
+metadata:
+  name: openshift-storage
+  labels:
+    kubernetes.io/metadata.name: openshift-storage
+spec: {}
+

bootstrap/roles/ocp4-install-noobaa/templates/odf-subscription.yaml.j2

@@ -0,0 +1,13 @@
+apiVersion: operators.coreos.com/v1alpha1
+kind: Subscription
+metadata:
+  labels:
+    operators.coreos.com/odf-operator.openshift-storage: ''
+  name: odf-operator
+  namespace: openshift-storage
+spec:
+  channel: {{ ocp4_channel }}
+  installPlanApproval: Automatic
+  name: odf-operator
+  source: redhat-operators
+  sourceNamespace: openshift-marketplace

bootstrap/roles/ocp4-install-noobaa/templates/operatorgroup-storage.yaml.j2

@@ -0,0 +1,8 @@
+apiVersion: operators.coreos.com/v1
+kind: OperatorGroup
+metadata:
+  name: openshift-storage-test
+  namespace: openshift-storage
+spec:
+  targetNamespaces:
+    - openshift-storage

bootstrap/roles/ocp4-install-quay/defaults/main.yaml

@@ -0,0 +1,29 @@
+quay_project_name: quay-demo
+quay_admin_username: quayadmin
+quay_admin_email: [email protected]
+quay_admin_password: quaypass123
+quay_registry_name: demo-registry
+quay_org_name: cicd-demo
+quay_secret_name: quay-robot-secret
+quay_repositories:
+  - spring-petclinic-dev
+  - spring-petclinic-stage
+quay_robot_account: demo_robot
+pipeline_namespace: cicd
+csrf_pattern: ".*window.__token\ =\ '(.*)';.*"
+#Can obtain status codes from Swagger of quay route/api/v1/discovery
+quay_user_found_success_status_code: 200
+quay_org_not_found_error_code: 404
+quay_org_found_success_status_code: 200
+quay_org_created_success_status_code: 201
+quay_repo_not_found_error_code: 404
+quay_repo_found_success_status_code: 200
+quay_repo_created_success_status_code: 201
+quay_robot_not_found_error_code: 400
+quay_robot_found_success_status_code: 200
+quay_robot_created_success_status_code: 201
+quay_perm_success_status_code: 200
+secret_required_namespaces:
+  - cicd
+  - devsecops-dev
+  - devsecops-qa