Camunda Optimize HELM Charts

Table of Contents

• Goals
• What is configured in this chart
• How does it work
• Steps to run this HELM chart
  • STOP before you run it
• Setup Infra and Architecture for Optimize
  • Step 0: Install HELM and Kubernetes
  • Step 1: Install Ingress Controller with Sticky Sessions
  • Step 2: Configure an External Elastic Search Instance
  • Step 3: Run Camunda
• Setup Optimize
  • Step 4: Configure the version of Optimize
  • Step 5: Configure Optimize Connections
  • Step 6: Configure the Optimize License
  • Troubleshoot volumes, configMaps and secrets
  • Step 7: Run the Chart
• Whats Next
• Based on Camunda Helm
• Project state

Goals

• Simple and understandable HELM Optimize example

• Help understand common configuration and architectural concepts

• Good documentation and example to assist in getting Optimize running quickly in Kubernetes

• NOT intended as a production ready configuration

What is configured in this chart

• ✓ Optimize

• ✓ Optimize EE Repo Pull

• ✓ Optimize License

• ✓ Load Balancer with sticky sessions

• ✓ Elastic Search

Note	for example of installing on AWS EKS see Install Optimize on EKS

How does it work

Important

The configs in this chart are preconfigured defaults and serve as a quick reference for understanding. You may need to change and adjust things to suit your use-case, infra, architecture, etc …​

• The HELM chart is an install descriptor to install Optimize on Kubernetes. HELM can do many things to help install and manage infra on Kubernetes.

• The primary configuration point is the values.yaml. It should allow you to get a basic Optimize configuration installed and running with little to no customizations.

• You still need to know how to debug on Kubernetes. See the Kubernetes Docs for help.

• While this chart defines how Optimize is installed other components need to be installed in your Kubernetes cluster to make Optimize work. See Setting up Infra to install the other components.

• You can find more on HELM here Helm Quickstart

Steps to run this HELM chart

STOP before you run it

Step 0. Install HELM and Kubernetes if not already installed

Step 1. Install Ingress Contorller to configure the loadbalaner ingress controller

Step 2. Install and Configure Elastic Search in the Kubernetes cluster

Step 3. Run Camunda you will need a Camunda instance Optimize can pull data from

Step 4. Configure Optimize Version

Step 5. Configure Optimize Connections to Camunda and Elastic Search

Step 6. Install Optimize License

Step 6. Run Optimize

Setup Infra and Architecture for Optimize

Step 0: Install HELM and Kubernetes

• You can find more on HELM here Helm Quickstart

• Kubernetes Getting Started

• Try Docker Desktop imo it’s the quickest way to get started with Kubernetes

• Tested with HELM

  • version.BuildInfo{Version:"v3.5.3", GitCommit:"041ce5a2c17a58be0fcd5f5e16fb3e7e95fea622", GitTreeState:"dirty", GoVersion:"go1.16"}

• Tested with Kubernetes

  • Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.2", GitCommit:"092fbfbf53427de67cac1e9fa54aaa09a28371d7", GitTreeState:"clean", BuildDate:"2021-06-16T12:59:11Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"darwin/amd64"}

  • Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.2", GitCommit:"092fbfbf53427de67cac1e9fa54aaa09a28371d7", GitTreeState:"clean", BuildDate:"2021-06-16T12:53:14Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"linux/amd64"}

---

Step 1: Install Ingress Controller with Sticky Sessions

Important

Kubernetes does not come with an implementation of a LoadBalancer or a Reverse Proxy for Ingress. The Ingerss resource allows you to configure a Controller for your needs. It’s important to understand what you need from an inrgess resource then you can choose the appropriate Controller to install. There are a variety of vendors.

Tested with https://kubernetes.github.io/ingress-nginx/deploy/#docker-for-mac

Install the NGINX Ingress Controller

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.44.0/deploy/static/provider/cloud/deploy.yaml

---

Configure the Ingress resource for Nginx with sticky sessions

Update the values.yaml and configure the Ingress Resource to tell the LoadBalancer (the NGINX deployment that was installed above) to stick to one Optimize instance once the user is logged into the webapps.

Defaults Below

  ingress:
    enabled: true
    annotations: {
        nginx.ingress.kubernetes.io/ingress.class: nginx,
        nginx.ingress.kubernetes.io/affinity: "cookie",
        nginx.ingress.kubernetes.io/affinity-mode: "persistent",
        nginx.ingress.kubernetes.io/session-cookie-expires: "172800",
        nginx.ingress.kubernetes.io/session-cookie-max-age: "172800",
      }
      # see more config options https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/
      # kubernetes.io/ingress.class: nginx
      # kubernetes.io/tls-acme: "true"
    hosts:
      - host: optimize.127.0.0.1.nip.io
        paths: ["/"]
    tls: []
    #  - secretName: camunda-optimize-tls
    #    hosts:
    #      - camunda-optimize.local   - camunda-optimize.local

---

Increase the replica count of the Nodes

Update the values.yaml and increase the replica count so the load balancer will send requests to both nodes for a user that is not already logged in to web apps.

Defaults Below

general:
  debug: true
  replicaCount: 2
  nameOverride: ""
  fullnameOverride: ""

---

Troubleshoot Kubernetes Resources

Troubleshoot Ingress, Pods and Services

• Check the Service and Ingress endpoints

  • kubectl describe ingress optimize-demo-camunda-optimize

  • kubectl describe service optimize-demo-camunda-optimize

• Check the pods

  • kubectl describe pods optimize-demo-camunda-optimize

• Check that the Service Selectors get the pods

  • kubectl get pods --show-labels | egrep 'app.kubernetes.io/instance=optimize-demo,app.kubernetes.io/name=camunda-optimize'

---

Step 2: Configure an External Elastic Search Instance

Download the Elastic HELM charts

Download with GIT: Elastic HELM Charts

Update the values.yaml Update the values.yaml in for the correct version of Elastic. Tested with 7.11.2 in the Docker for Mac Examples.

Use Values Below

# Permit co-located instances for solitary minikube virtual machines.
antiAffinity: "soft"

# Shrink default JVM heap.
esJavaOpts: "-Xmx128m -Xms128m"

imageTag: "7.11.2"

# Allocate smaller chunks of memory per pod.
resources:
  requests:
    cpu: "100m"
    memory: "512M"
  limits:
    cpu: "1000m"
    memory: "512M"

# Request smaller persistent volumes.
volumeClaimTemplate:
  accessModes: [ "ReadWriteOnce" ]
  storageClassName: "hostpath"
  resources:
    requests:
      storage: 1G

Run the install

make install

Test the install

  kubectl port-forward svc/elasticsearch-master 9200

  curl localhost:9200/_cat/indices

Step 3: Run Camunda

To Run Camunda see the

• HELM Chart to install on Kubernetes

• or Download a Distro or use SpringBoot

• or Run in Docker

Setup Optimize

Step 4: Configure the version of Optimize

Important

Get an enterprise trial to try Optimize

In this case the latest image is used. But we could swap different images and versions.

See the Camunda Harbor Repo if you need a different version of Optimize.

Pulling from the Enterprise Repo

Note	you will need your enterprise credentials and an enterprise license for Optimize.

The version of Optimize can be changed in this section of the values.yaml

Defaults Below

image:
  repository: registry.camunda.cloud/optimize-ee/optimize
  tag: latest
  pullPolicy: IfNotPresent
  pullSecrets:
    - name: camunda-reg-cred

---

Note	If issues arise with pulling the image the workaround is to manually pull the image. Run the following commands

docker login registry.camunda.cloud

docker pull registry.camunda.cloud/optimize-ee/optimize:latest

---

Configuring the pullSecrets

Tip	see Configuring pull secrets kubernetes doc and Managing Secrets for more info

Install the secret and name it camunda-reg-cred

kubectl create secret docker-registry camunda-reg-cred --docker-server=registry.camunda.cloud --docker-username=<<user>> --docker-password=<<password>> --docker-email=<your-email>

Tip	You may need to deref special characters in your passwords i.e. --docker-password=mypassword\!isstrong

Check your secret

kubectl get secret camunda-reg-cred --output=yaml

kubectl get secret camunda-reg-cred --output="jsonpath={.data.\.dockerconfigjson}" | base64 --decode

---

Step 5: Configure Optimize Connections

Configure the connection to Camunda

See Running Camunda to ensure you have a running Camunda instance.

Important

You need to do this step if you did not use the Camunda-HELM project to install Camunda BPM and you defined your own Camunda BPM deployment and services.

Go to values.yaml to update configs. Make sure to update the restUrl and webappsUrl with the correct config from the ClusterIP Service you’ve defined.

Replace <workflow-demo-camunda-bpm-platform> with the corresponding Service Endpoint.

You can find your correct Camunda BPM endpoint by checking the Service you defined.

kubectl get service

Again, understand that you only need to change this value if you defined your own Camunda BPM deployments and services. If you used the Camunda-HELM project to install Camunda BPM then you do not need to change the config.

Defaults Below

# Optimize Camunda Platform connection configurations
# These have not yet been applied in the data-environment.yaml
camunda:
  engine:
    name: "default"
    webappsEnabled: true
    restUrl: "http://workflow-demo-camunda-bpm-platform:8080/engine-rest"
    webappsUrl: "http://workflow-demo-camunda-bpm-platform:8080/camunda"
    importEnabled: true
    eventImportEnabled: true
    authentication:
      enabled: false
      user: ''
      password: ''

Optimize Specific Configs

Many things can be configured in optimize it’s possible to extend the HELM chart to support these configs by updating the data-environment.yaml and corresponding values.yaml configs

# Optimize specific configurations
# These have not yet been applied in the data-environment.yaml
optimize:
  local:
    available:
    fallback:
  history:
    cleanup:
      cronTrigger:
      ttl:
  webhook:
    alerting:
      '${webhookName}':
          url:
  email:
    enabled:
  eventBasedProcess:
    eventImport:
      enabled:
  telemetry:
    initializeTelemetry:
  sharing:
    enabled:
  plugin:
    directory:

Tip	See the additional configurations for Optimize

ConfigMap for Optimize Configurations

The environment-config ConfigMap defined in data-environment.yaml applies the configs to Optimize This will be mounted as a file in the Pod in the config directory as environment-config.yaml where Optimize can pick it up. If you wish to add additional configs you can use this config map and expose the variables in the values.yaml

Warning

The es.connection.nodes.host and httpPort are not affective. Instead I use the environment vars in the deployment.yml to set these environment variables. This is a bit of a hack and will be fixed soon.

name: OPTIMIZE_ELASTICSEARCH_HOST
value: "{{ .Values.elasitc.connect.host }}"
name: OPTIMIZE_ELASTICSEARCH_HTTP_PORT
value: "{{ .Values.elasitc.connect.port }}"

Defaults Below

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    {{- include "camunda-optimize.labels" . | nindent 4 }}
  name: environment-config
data:
    environment-config.yaml: |
        engines:
          'camunda-bpm':
            name: {{ .Values.camunda.engine.name }}
            rest: {{ .Values.camunda.engine.restUrl }}
            importEnabled: {{ .Values.camunda.engine.importEnabled }}
            eventImportEnabled: {{ .Values.camunda.engine.eventImportEnabled }}
            webapps:
              endpoint: {{ .Values.camunda.engine.webappsUrl }}
              enabled: {{ .Values.camunda.engine.webappsEnabled }}
            authentication:
              enabled: {{ .Values.camunda.engine.authentication.enabled }}
              user: {{ .Values.camunda.engine.authentication.user }}
              password: {{ .Values.camunda.engine.authentication.password }}
        es:
          connection:
            timeout: {{ .Values.elasitc.connect.timeout }}
            nodes:
              host: {{ .Values.elasitc.connect.host }}
              httpPort: {{ .Values.elasitc.connect.port }}
          proxy:
            enabled: {{ .Values.elasitc.connect.proxy.enabled }}
            host: {{ .Values.elasitc.connect.proxy.host }}
            port: {{ .Values.elasitc.connect.proxy.port }}
            sslEnabled: {{ .Values.elasitc.connect.proxy.sslEnabled }}

---

Configure Connection to Elastic Search

See values.yaml to update configs. You wont need to change unless you change the Elastic install.

Defaults Below

# Optimize Elastic connection configurations
# These have not all yet been applied in the data-environment.yaml
elasitc:
  connect:
    timeout: 10000
    credentialsSecertName: "elastic-search-credentials"
    host: "elasticsearch-master"
    port: "9200"
    proxy:
      enabled: false
      host: "localhost"
      port: 80
      sslEnabled: false
    security:
      username:
      password:
      ssl:
        enabled: false
        certificate:
        certificate_authorities: []

Environment configs mounting definition in the deployment.yaml This is informational. Nothing to do unless you want to change the mount location or type.

          volumeMounts:
          - mountPath: /optimize/config/environment-config.yaml
            subPath: environment-config.yaml
            name: environment-config

      volumes:
      - name: environment-config
        configMap:
          name: environment-config

Step 6: Configure the Optimize License

Important

Get an enterprise trial to try Optimize

Add your license to the data-license.yaml

Defaults Below

apiVersion: v1
kind: Secret
metadata:
  labels:
    {{- include "camunda-optimize.labels" . | nindent 4 }}
  name: optimize-license
stringData:
  OptimizeLicense.txt:
    --------------- BEGIN CAMUNDA LICENSE KEY ---------------

    ---------------  END CAMUNDA LICENSE KEY  ---------------

The license Secret mounting definition in the deplyment.yaml This is informational. Nothing to do unless you want to change the mount location or type.

          volumeMounts:
          - mountPath: /optimize/config/OptimizeLicense.txt
            subPath: OptimizeLicense.txt
            name: optimize-license

      volumes:
      - name: optimize-license
        secret:
          secretName: optimize-license

Troubleshoot volumes, configMaps and secrets

Check the secret exists

kubectl get secret optimize-license -o yaml

kubectl get secret camunda-reg-cred --output=yaml

Check the secrets are created properly

kubectl get secret camunda-reg-cred --output="jsonpath={.data.\.dockerconfigjson}" | base64 --decode

kubectl get secret optimize-license --output="jsonpath={.data.OptimizeLicense\.txt}"

Look at the mounted license file use exec command into pod file system. You should see

cat config/OptimizeLicense.txt

See Managing Secrets for more info.

Tip	For more configuration options see https://github.com/camunda/docker-camunda-optimize/blob/next/README.md

Step 7: Run the Chart

Runing the Chart the following command to install the chart and apply the configurations to the Kubernetes cluster

helm install optimize-demo ./charts/camunda-optimize/

Change the Chart  — When you make changes run the following command to apply the changes to the cluster

helm upgrade optimize-demo ./charts/camunda-optimize/

Remove the Chart  — To remove the installation

helm uninstall optimize-demo

Whats Next

• ✓ Configure Ingress and Scaling

• ✓ Configuration for EE License (In Progress)

• ✓ Configure common Optimize configs (Elastic, Engine)

• ❏ Configuration for Secrets Vault (HashiCorp, Spring Cloud Vault)

• ❏ Configuration for LDAP plugin

• ❏ Adding an Engine plugin

• ❏ Configuration for Logging

  • ❏ Configuration for Log Drain

  • ❏ Configuration for ARGO

  • ❏ Configuration for TERRAFORM

• ❏ Configurations for SSO

  • ❏ with Keycloak

• ❏ Configure auto-scaling

• ❏ Configure Cloud Deployments (GKE, AWS, Azure)

Based on Camunda Helm

More Info

• https://docs.camunda.org/optimize/develop/technical-guide

• https://artifacthub.io/packages/helm/camunda/camunda-optimize

• https://github.com/elastic/helm-charts

• https://registry.camunda.cloud/harbor/projects/4/repositories

• https://kubernetes.github.io/ingress-nginx/deploy/#docker-for-mac

• https://helm.sh/docs/intro/quickstart/

• https://hub.docker.com/r/camunda/camunda-optimize

• https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/

• https://github.com/camunda/camunda-helm

• https://github.com/camunda/docker-camunda-optimize

Project state

This project is in alpha phase.