Enhancements and Basic OCP Support

docs/CONFIG-VARS.md

@@ -26,6 +26,7 @@ Supported configuration variables are listed in the table below.  All variables
   - [CONNECT](#connect)
   - [Workload Orchestrator](#workload-orchestrator)
   - [Miscellaneous](#miscellaneous)
+  - [Openshift](#openshift)
   - [Third-Party Tools](#third-party-tools)
     - [Cert-manager](#cert-manager)
     - [Cluster Autoscaler](#cluster-autoscaler)
@@ -35,6 +36,7 @@ Supported configuration variables are listed in the table below.  All variables
     - [NFS Client](#nfs-client)
     - [Postgres NFS Client](#postgres-nfs-client)
 
+
 ## BASE
 
 | Name | Description | Type | Default | Required | Notes | Tasks |
@@ -365,6 +367,29 @@ Notes:
     - For example, defining `V4_CFG_VIYA_STOP_SCHEDULE` and not `V4_CFG_VIYA_START_SCHEDULE` will result in a Viya stop job that runs on a schedule and a suspended Viya start job that you will be able to manually trigger.
   - Defining both `V4_CFG_VIYA_START_SCHEDULE` and `V4_CFG_VIYA_STOP_SCHEDULE` will result in a non-suspended Viya start and stop job that runs on the schedule you defined.
 
+## Openshift
+
+This tool has basic support for Openshift. It will configure routes, TLS, remove seccomp profile, and can **optionally** setup SecurityContextConstraints (SCC) and dependant RBAC permissions. 
+
+Notes:
+  - <V4_CFG_INGRESS_TYPE> must be set to "route"
+  - This tool does **not** handle the configurations needed for sas-opendistro or OpenSearch needed when running on Openshift.
+
+| Name | Description | Type | Default | Required | Notes | Tasks |
+| :--- | ---: | ---: | ---: | ---: | ---: | ---: |
+| V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED | Globally setting to enable SCC configuration and dependent RBAC creation | bool | false | false | Can be overridden with individual settings below | viya |
+| V4_CFG_OPENSHIFT_SCC_CAS_SERVER_MODE | CAS server mode | string | standard | false | Supported values: [`standard`, `sssd`, `host`] | viya |
+| V4_CFG_OPENSHIFT_SCC_AIRFLOW | Configure SCC and dependent RBAC for sas-airflow | bool | <V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED> | false | Only applicable if deployment includes SAS Airflow | viya |
+| V4_CFG_OPENSHIFT_SCC_ESP_PROJECT | Configure SCC and dependent RBAC for sas-esp-project | bool | <V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED> | false | Only applicable if deployment includes SAS Event Stream Processing | viya |
+| V4_CFG_OPENSHIFT_SCC_MICROANALYTIC_SCORE | Configure SCC and dependent RBAC for sas-microanalytic-score | bool | <V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED> | false | Only applicable if deployment includes SAS Microanalytic Score | viya |
+| V4_CFG_OPENSHIFT_SCC_MODEL_PUBLISH_KANIKO | Configure SCC and dependent RBAC for sas-model-publish-kaniko | bool | <V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED> | false | Required to be set to **true** if you plan to publish models with SAS Model Manager or SAS Intelligent Decisioning to containers using kaniko | viya |
+| V4_CFG_OPENSHIFT_SCC_MODEL_REPOSITORY | Configure SCC and dependent RBAC for sas-model-repository | bool | <V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED> | false | Only applicable if deployment includes SAS Model Repository | viya |
+| V4_CFG_OPENSHIFT_SCC_PYCONFIG | Configure SCC and dependent RBAC for sas-pyconfig | bool | <V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED> | false | | viya |
+| V4_CFG_OPENSHIFT_SCC_PROGRAMMING_ENVIRONMENT | Configure SCC and dependent RBAC for sas-programming-environment | bool | <V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED> | false | This or <V4_CFG_OPENSHIFT_SCC_WATCHDOG> should be set true but not both | viya |
+| V4_CFG_OPENSHIFT_SCC_PROGRAMMING_ENVIRONMENT_HOSTPATH | Does this deployment use hostPath volume mounts | bool | false | false | | viya |
+| V4_CFG_OPENSHIFT_SCC_CONNECT_SPAWNER | Configure SCC and dependent RBAC for sas-connect-spawner | bool | <V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED> | false | Required to be set to **true** only if you intend to launch your SAS/CONNECT servers in the Spawner pod, rather than in their own pods | viya |
+| V4_CFG_OPENSHIFT_SCC_WATCHDOG | Configure SCC and dependent RBAC for sas-watchdog | bool | false | false | This or <V4_CFG_OPENSHIFT_SCC_PROGRAMMING_ENVIRONMENT> should be set true but not both. <br>Required to be set to **true** if you are deploying SAS Watchdog | viya |
+
 ## Third-Party Tools
 
 ### Cert-manager
@@ -456,4 +481,4 @@ The Postgres NFS client is currently supported by the nfs-subdir-external-provis
 | PG_NFS_CLIENT_CHART_URL | nfs-subdir-external-provisioner Helm chart URL | string | Go [here](https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/) for more information. | false | | baseline |
 | PG_NFS_CLIENT_CHART_NAME | nfs-subdir-external-provisioner Helm chart name | string | nfs-subdir-external-provisioner | false | | baseline |
 | PG_NFS_CLIENT_CHART_VERSION | nfs-subdir-external-provisioner Helm chart version | string | 4.0.18| false | | baseline |
-| PG_NFS_CLIENT_CONFIG | nfs-subdir-external-provisioner Helm values | string | See [this file](../roles/baseline/defaults/main.yml) for more information. | false | | baseline |
+| PG_NFS_CLIENT_CONFIG | nfs-subdir-external-provisioner Helm values | string | See [this file](../roles/baseline/defaults/main.yml) for more information. | false | | baseline |

roles/vdm/defaults/main.yaml

@@ -117,3 +117,17 @@ V4_WORKLOAD_ORCHESTRATOR_ENABLED: true
 
 ## NIST Features
 V4_CFG_NIST_FEATURES_ENABLED: false
+
+## Openshift
+V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED: false
+V4_CFG_OPENSHIFT_SCC_CAS_SERVER_MODE: standard # standard, ssd, host 
+V4_CFG_OPENSHIFT_SCC_AIRFLOW: "{{ V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED }}"
+V4_CFG_OPENSHIFT_SCC_ESP_PROJECT: "{{ V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED }}"
+V4_CFG_OPENSHIFT_SCC_MICROANALYTIC_SCORE: "{{ V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED }}"
+V4_CFG_OPENSHIFT_SCC_MODEL_PUBLISH_KANIKO: "{{ V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED }}"
+V4_CFG_OPENSHIFT_SCC_MODEL_REPOSITORY: "{{ V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED }}"
+V4_CFG_OPENSHIFT_SCC_PYCONFIG: "{{ V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED }}"
+V4_CFG_OPENSHIFT_SCC_PROGRAMMING_ENVIRONMENT: "{{ V4_CFG_OPENSHIFT_SCC_RBAC_ENABLED if not V4_CFG_OPENSHIFT_SCC_WATCHDOG else false }}"
+V4_CFG_OPENSHIFT_SCC_PROGRAMMING_ENVIRONMENT_HOSTPATH: false
+V4_CFG_OPENSHIFT_SCC_CONNECT_SPAWNER: false 
+V4_CFG_OPENSHIFT_SCC_WATCHDOG: false

roles/vdm/library/overlay_facts.py

@@ -30,46 +30,34 @@ def main():
     module.exit_json(**results)
 
   try:
-    if len(module.params['add']) > 0:
-      for overlay in module.params['add']:
-
-        # Version checks
-        minVersion = parse_version(str(overlay.setdefault("min", "0.0.0")))
-        if "max" in overlay and module.params["cadence_name"].lower() == "fast":
-          continue
-        maxVersion = parse_version(str(overlay.setdefault("max", "9999.9999.9999")))
-        existingVersion = parse_version(module.params['cadence_number'])
-        if ((existingVersion < minVersion) and module.params["cadence_name"].lower() != "fast") or (existingVersion > maxVersion):
-          continue
+    for overlay in module.params['add']:
+
+      # Version checks
+      minVersion = parse_version(str(overlay.setdefault("min", "0.0.0")))
+      if "max" in overlay and module.params["cadence_name"].lower() == "fast":
+        continue
+      maxVersion = parse_version(str(overlay.setdefault("max", "9999.9999.9999")))
+      existingVersion = parse_version(module.params['cadence_number'])
+      if ((existingVersion < minVersion) and module.params["cadence_name"].lower() != "fast") or (existingVersion > maxVersion):
+        continue
 
-        priority = str(overlay.setdefault("priority", 1))
-        phase = "pre" if int(priority) < 50 else "post"
-        overlay.pop("priority", None)
-        overlay_type = list(overlay.keys())[0]
+      priority = str(overlay.setdefault("priority", 10))
+      overlay.pop("priority", None)
+      overlay_type = list(overlay.keys())[0]
 
-        # set correct path for vdm or sas-bases patches
-        folderPath = os.path.join("site-config/vdm", overlay_type) if bool(overlay.setdefault("vdm", False)) else "sas-bases/"
-        overlay_path = os.path.join(folderPath, overlay[overlay_type])
-        
-        module.params['existing'].setdefault(overlay_type, {})
-        module.params['existing'][overlay_type].setdefault(phase, {})
+      # set correct path for vdm or sas-bases patches
+      folderPath = "sas-bases/" if ("/" in overlay[overlay_type] or overlay[overlay_type] == "base") else os.path.join("site-config/vdm", overlay_type)
+      overlay_path = os.path.join(folderPath, overlay[overlay_type])
+
+      module.params['existing'].setdefault(overlay_type, {})
+      module.params['existing'][overlay_type].setdefault(priority, [])
 
-        if priority in module.params['existing'][overlay_type][phase]:
-          if overlay_path not in module.params['existing'][overlay_type][phase][priority]:
-            module.params['existing'][overlay_type][phase][priority].append(overlay_path)
-        else:
-          module.params['existing'][overlay_type][phase].update({priority: [overlay_path]})
+      # duplicate check
+      if overlay_path not in module.params['existing'][overlay_type][priority]:
+        module.params['existing'][overlay_type][priority].append(overlay_path)
 
-      results['ansible_facts'] = {"vdm_overlays": module.params['existing']}
-      module.exit_json(**results)
-    else:
-      for resource_type, phases in module.params['existing'].items():
-        results['result'][resource_type] = {}
-        for phase in phases:
-          results['result'][resource_type][phase] = []
-          for priority in sorted(module.params['existing'][resource_type][phase]):
-            results['result'][resource_type][phase] += module.params['existing'][resource_type][phase][priority]
-      module.exit_json(**results)
+    results['ansible_facts'] = {"vdm_overlays": module.params['existing']}
+    module.exit_json(**results)
   except Exception as e:
     module.fail_json(error=e, msg="Error occurred")
     raise

roles/vdm/library/siteconfig_info.py

@@ -6,6 +6,7 @@
 import glob
 import yaml
 import os
+import re
 from enum import Enum, auto
 
 class Overlay(Enum):
@@ -16,15 +17,22 @@ class Overlay(Enum):
   TRANSFORMER = "transformers"
 
 class siteConfig(object):
-  def __init__(self, basedir):
-    self._overlays = dict()
+  def __init__(self, basedir, overlays, fail_json):
+    self._overlays = overlays
     self._basedir = os.path.join(basedir, '')
+    self._fail_json = fail_json
 
   def add_overlays(self, overlay_type: Overlay, config: str):
-    if overlay_type.value in self._overlays:
-      self._overlays[overlay_type.value].append(self.remove_basedir(config))
-    else:
-      self._overlays[overlay_type.value] = [self.remove_basedir(config)]
+    match = re.search(r"^(\d{1,2})", os.path.basename(config))
+    priority = 50
+    if match:
+      if int(match.group(1)) == 0:
+        self._fail_json(f"User defined overlay cannot be priority 0, {config}")
+      else:
+        priority = int(match.group(1))
+    self._overlays.setdefault(overlay_type.value, {})
+    self._overlays[overlay_type.value].setdefault(priority, [])
+    self._overlays[overlay_type.value][priority].append(self.remove_basedir(config))
 
   def get_overlays(self) -> dict:
     return self._overlays
@@ -104,10 +112,11 @@ def main():
   fields = {
     "path": {"required": True, "type": "str"},
     "exclude": {"default": [], "type": list},
+    "existing": {"default": {}, "type": dict}
   }
   module = AnsibleModule(argument_spec=fields)
   try:
-    sc = siteConfig(module.params['path'])
+    sc = siteConfig(module.params['path'], module.params['existing'], module.fail_json)
     scFolder = os.path.join(module.params['path'], 'site-config')
     _, folders, _ = next(os.walk(scFolder))
     for folder in folders:

roles/vdm/tasks/cas.yaml

@@ -39,16 +39,16 @@
         existing: "{{ vdm_overlays }}"
         add:
           - { transformers: overlays/cas-server/cas-sssd-sidecar.yaml, priority: 0 }
-          - { transformers: cas-sssd.yaml, vdm: true }
-          - { generators: sas-sssd-config.yaml, vdm: true }
+          - { transformers: cas-sssd.yaml }
+          - { generators: sas-sssd-config.yaml }
 
 - name: CAS - backup controller
   overlay_facts:
     cadence_name: "{{ V4_CFG_CADENCE_NAME }}"
     cadence_number: "{{ V4_CFG_CADENCE_VERSION }}"
     existing: "{{ vdm_overlays }}"
     add:
-      - { transformers: cas-manage-backup.yaml, vdm: true }
+      - { transformers: cas-manage-backup.yaml }
   when:
     - V4_CFG_CAS_ENABLE_BACKUP_CONTROLLER
   tags:
@@ -77,7 +77,7 @@
     cadence_number: "{{ V4_CFG_CADENCE_VERSION }}"
     existing: "{{ vdm_overlays }}"
     add:
-      - { transformers: cas-manage-cpu-and-memory.yaml, vdm: true }
+      - { transformers: cas-manage-cpu-and-memory.yaml }
   when:
     - V4_CFG_CAS_RAM is not none
     - V4_CFG_CAS_CORES is not none
@@ -92,7 +92,7 @@
     cadence_number: "{{ V4_CFG_CADENCE_VERSION }}"
     existing: "{{ vdm_overlays }}"
     add:
-      - { transformers: cas-manage-workers.yaml, vdm: true }
+      - { transformers: cas-manage-workers.yaml }
   when:
     - V4_CFG_CAS_WORKER_COUNT |int > 1
   tags:
@@ -106,7 +106,7 @@
     cadence_number: "{{ V4_CFG_CADENCE_VERSION }}"
     existing: "{{ vdm_overlays }}"
     add:
-      - { transformers: cas-enable-external-services.yaml, vdm: true }
+      - { transformers: cas-enable-external-services.yaml }
   when:
     - V4_CFG_CAS_ENABLE_LOADBALANCER
   tags:
@@ -120,7 +120,7 @@
     cadence_number: "{{ V4_CFG_CADENCE_VERSION }}"
     existing: "{{ vdm_overlays }}"
     add:
-      - { transformers: cas-auto-restart.yaml, vdm: true, min: "2021.2" }
+      - { transformers: cas-auto-restart.yaml, min: "2021.2" }
   when:
     - V4_DEPLOYMENT_OPERATOR_ENABLED
     - V4_CFG_CAS_ENABLE_AUTO_RESTART

roles/vdm/tasks/connect.yaml

@@ -8,7 +8,7 @@
     cadence_number: "{{ V4_CFG_CADENCE_VERSION }}"
     existing: "{{ vdm_overlays }}"
     add:
-      - { resources: sas-connect-spawner-enable-loadbalancer.yaml, vdm: true }
+      - { resources: sas-connect-spawner-enable-loadbalancer.yaml }
   when:
     - V4_CFG_CONNECT_ENABLE_LOADBALANCER
   tags:

roles/vdm/tasks/helpers/add_if_exists.yaml

@@ -0,0 +1,56 @@
+# Copyright © 2020-2024, SAS Institute Inc., Cary, NC, USA. All Rights Reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+---
+- name: Check if file exists - {{ resource }}
+  stat:
+    path: "{{ DEPLOY_DIR }}/sas-bases/{{ resource }}"
+  register: resource_check
+  tags:
+    - install
+    - uninstall
+    - update
+
+- block:
+  - set_fact:
+      copy_overlay_item: "{{ lookup('template', 'copy_if_exists.yaml.j2') | from_yaml }}"
+    loop : "{{ overlays |flatten(levels=1) }}"
+    register: copy_overlay_result
+
+  - set_fact: 
+      copy_overlays: "{{ copy_overlay_result.results | map(attribute='ansible_facts.copy_overlay_item') | list }}"
+    tags:
+      - install
+      - uninstall
+      - update
+
+  - copy:
+      src: "{{ DEPLOY_DIR }}/sas-bases/{{ item.path }}"
+      dest: "{{ role_path }}/templates/{{ item.type }}/"
+      mode: "0660"
+    with_items:
+      - "{{ copy_overlays }}"
+    when: "item.path.startswith('examples')"
+    tags:
+      - install
+      - uninstall
+      - update
+
+  - overlay_facts:
+      cadence_name: "{{ V4_CFG_CADENCE_NAME }}"
+      cadence_number: "{{ V4_CFG_CADENCE_VERSION }}"
+      existing: "{{ vdm_overlays }}"
+      add:
+        - "{{ [[item.type, item.filename], ['priority', item.priority]] | community.general.dict  }}"
+    with_items:
+      - "{{ copy_overlays }}"
+    tags:
+      - install
+      - uninstall
+      - update
+
+  when: resource_check.stat.exists
+  tags:
+    - install
+    - uninstall
+    - update

roles/vdm/tasks/kustomize.yaml

@@ -29,41 +29,33 @@
     cadence_number: "{{ V4_CFG_CADENCE_VERSION }}"
     existing: "{{ vdm_overlays }}"
     add:
-      - { resources: sas-deployment-buildinfo.yaml, vdm: true }
+      - { resources: sas-deployment-buildinfo.yaml }
   tags:
     - install
     - uninstall
     - update
 
 - name: Kustomize - Get user's customizations
   siteconfig_info:
+    existing: "{{ vdm_overlays }}"
     path: "{{ DEPLOY_DIR }}"
     exclude:
       - vdm
       - multi-tenant
       - sas-tenant-job
       - operator-deploy
-  register: user_customizations
-  tags:
-    - install
-    - uninstall
-    - update
-
-- name: Kustomize - Ordered overlays
-  overlay_facts:
-    existing: "{{ vdm_overlays }}"
-  register: ordered_overlays
+  register: unified_overlays
   tags:
     - install
     - uninstall
     - update
 
 - name: Kustomize - Copy VDM overlays
-  include_tasks: copy_overlay.yaml
+  include_tasks: helpers/copy_overlay.yaml
   vars:
     resource: "{{ item.key }}"
-    overlays: "{{ item.value.pre | default([], true) + item.value.post | default([], true) }}"
-  with_dict: "{{ ordered_overlays.result }}"
+    overlays: "{{ item.value.values() | default([], true) }}"
+  with_dict: "{{ unified_overlays['overlays'] }}"
   tags:
     - install
     - uninstall