Large Language Model (LLM) on OpenShift

The intent of this repository is to automate the deployment of a large language model and its supporting components on OpenShift.

Prerequisites

This process assumes you already have an OpenShift 4.12+ running with cluster admin permissions.

You will also need the OpenShift client oc.

Step 1 - Install openshift gitops

The goal of this setup process is to be as declarative as possible. With this in mind our first step on the new cluster will be to install the OpenShift GitOps operator and create an instance of ArgoCD via the operator, so that all remaining steps can be performed in a GitOps manner.

1.1 Login to the cluster

Before we can run any of the following commands, we need to ensure we are logged in. Run the following to do so, updating the placeholder values:

oc login --token=<token> \
         --server=<cluster> 
         --insecure-skip-tls-verify=true

1.2 Install openshift gitops operator

We can programatically install the openshift gitops operator on the cluster in a declaritive way by creating a Subscription kubernetes custom resource to subscribe a given namespace to the Operator.

cat << EOF | oc apply --filename -
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: openshift-gitops-operator
  namespace: openshift-operators
spec:
  channel: latest
  installPlanApproval: Automatic
  name: openshift-gitops-operator
  source: redhat-operators
  sourceNamespace: openshift-marketplace
  config:
    env:
    - name: ARGOCD_CLUSTER_CONFIG_NAMESPACES
      value: openshift-gitops
EOF

1.3 Update openshift gitops instance rbac

Once the operator is installed we can apply our ArgoCD custom resource definition. This will be picked up by the operator and an updated argocd instance will be deployed based on the specification we provided.

The updates below are primarily to rbac, so that we can login via OpenShift SSO and still see applications targeting our local cluster.

cat << EOF | oc apply --filename -
apiVersion: argoproj.io/v1beta1
kind: ArgoCD
metadata:
  name: openshift-gitops
  namespace: openshift-gitops
spec:
  kustomizeBuildOptions: --enable-helm --enable-alpha-plugins
  rbac:
    defaultPolicy: role:admin
    scopes: '[groups]'
  applicationSet: {}
  resourceExclusions: |
    - kinds:
        - TaskRun
  server:
    insecure: true
    route:
      enabled: true
      tls:
        insecureEdgeTerminationPolicy: Redirect
        termination: edge
  sso:
    dex:
      openShiftOAuth: true
    provider: dex
EOF

Once the argocd instance has started we can access the web interface via the Route automatically created by the Operator and click “Login with OpenShift”.

echo "https://$(oc --namespace openshift-gitops get route openshift-gitops-server --output jsonpath='{.spec.host}')"

Step 2 - Create gitops applicationset

REST REMOVED