Add support for cri-resource-manager project

This commit adds support for installing and configuring the [cri-resource-manager](https://github.com/intel/cri-resource-manager) project as an systemd-based service. It also adds the automation to configure `kubelet` service to consume it as a remote container runtime. Finally, it's providing the automation for cleaning up the `kubelet` service configuration to its original state without `cri-resource-manager`. Binary installation will be temporally consumed from a personal fork that is currently hosting `cri-resource-manager` binaries in the meantime that `cri-resource-manager` generates its packaging strategy. Signed-off-by: Obed N Munoz <[email protected]>
clearlinux · Oct 23, 2019 · 623b4b4 · 623b4b4
1 parent f2e8941
commit 623b4b4
Show file tree

Hide file tree

Showing 5 changed files with 134 additions and 0 deletions.
diff --git a/clr-k8s-examples/10-cri-resource-manager/README.md b/clr-k8s-examples/10-cri-resource-manager/README.md
@@ -0,0 +1,58 @@
+CRI Resource Manager
+====================
+CRI Resource Manager serves as a relay/proxy between kubelet and the container runtime,
+relaying requests and responses back and forth between these two, potentially alrering requests
+as they fly by.
+
+This document explains a very simple use case for the `cri-resource-manager`, for more details and tweaks
+on CRI Resource Manager service, you can go to https://github.com/intel/cri-resource-manager.
+
+Install
+-------
+[`install.sh`](install.sh) script will download the binary and install it as an `systemd` service unit. Below you can see the available
+variables you can use to customize the usage of your CRI Resource Manager service.
+
+| Variable                    | Description                               | Default Value                                    |
+|-----------------------------|-------------------------------------------|--------------------------------------------------|
+| `RUNNER`                    | Default Container Runtime                 | `containerd`                                     |
+| `CRI_RESMGR_POLICY`         | CRI Resource Manager Policy type          | `null`                                           |
+| `CRI_RESMGR_POLICY_OPTIONS` | CRI Resource Manager extra policy options | `-dump='reset,full:.*' -dump-file=/tmp/cri.dump` |
+| `CRI_RESMGR_DEBUG_OPTIONS`  | CRI Resource Manager debugging options    |                                                  |
+
+```
+RUNNER=containerd ./install.sh
+```
+
+- Install verification
+  - Verify that the cri-resource-manager service is actually running.
+```
+systemctl status cri-resource-manager
+```
+  - Verify that the `/var/run/cri-resmgr/cri-resmgr.sock` is created, it will indicate that `cri-resource-manager` is ready to receive requests.
+
+
+Setup as a container runtime in `kubelet`
+----------------------------------------
+The [`setup.sh`](setup.sh) script will configure the `kubelet` service to use the `cri-resource-manager` relay as its remote container runtime.
+```
+./setup.sh
+```
+
+- Setup verification
+  - Kubelet service should be restarted and now using `cri-resource-manager` as its container runtime
+  - `cri-resource-manager` service's logs will be located at `/tmp/cri.dump`
+```
+tail /tmp/cri.dump
+```
+
+Cleanup
+-------
+The [`clean.sh`](clean.sh) will first clean the `kubelet` service as it was before the `cri-resource-manager` and restarts `kubelet` service.
+Then. it will proceed to stop the `cri-resource-manager` service.
+```
+./clean.sh
+```
+
+More kubernetes native approach (experimental)
+----------------------------------------------
+In case that you're interested in a more Kubernetes native way of deploying the CRI Resource manager, take a look on: https://github.com/obedmr/cri-resource-manager/blob/cloud-native/cmd/cri-resmgr/deployment.yaml
diff --git a/clr-k8s-examples/10-cri-resource-manager/clean.sh b/clr-k8s-examples/10-cri-resource-manager/clean.sh
@@ -0,0 +1,18 @@
+#!/bin/bash
+
+set -o errexit
+set -o nounset
+
+# Kubelet
+KUBEADM_FLAGS="/var/lib/kubelet/kubeadm-flags.env"
+sudo rm -f /etc/systemd/system/kubelet.service.d/99-cri-resource-manager.conf
+sudo systemctl daemon-reload
+sudo systemctl restart kubelet
+
+if sudo test -f "$KUBEADM_FLAGS.bkp" ; then
+    sudo mv $KUBEADM_FLAGS.bkp $KUBEADM_FLAGS
+fi
+
+# CRI Resource Manager
+sudo systemctl stop cri-resource-manager
+sudo systemctl disable cri-resource-manager
diff --git a/clr-k8s-examples/10-cri-resource-manager/install.sh b/clr-k8s-examples/10-cri-resource-manager/install.sh
@@ -0,0 +1,31 @@
+#!/bin/bash
+
+set -o errexit
+set -o nounset
+
+RUNNER=${RUNNER:-"containerd"}
+CRI_RESMGR_POLICY=${CRI_RESMGR_POLICY:-"null"}
+CRI_RESMGR_POLICY_OPTIONS=${CRI_RESMGR_POLICY_OPTIONS:-"-dump='reset,full:.*' -dump-file=/tmp/cri.dump"}
+CRI_RESMGR_DEBUG_OPTIONS=${CRI_RESMGR_DEBUG_OPTIONS:-""}
+
+curl https://raw.githubusercontent.com/obedmr/cri-resource-manager/master/godownloader.sh | bash
+sudo cp ./bin/* /usr/bin/
+
+runtime_socket=$(sudo find /run/ -iname $RUNNER.sock | head -1)
+CRI_RESMGR_POLICY_OPTIONS+=" -runtime-socket=$runtime_socket -image-socket=$runtime_socket"
+
+sudo mkdir -p /etc/sysconfig/
+cat <<EOF | sudo tee /etc/sysconfig/cri-resource-manager
+POLICY=$CRI_RESMGR_POLICY
+POLICY_OPTIONS=$CRI_RESMGR_POLICY_OPTIONS
+DEBUG_OPTIONS=$CRI_RESMGR_DEBUG_OPTIONS
+EOF
+
+sudo mkdir -p /etc/systemd/system/
+curl https://raw.githubusercontent.com/obedmr/cri-resource-manager/master/cmd/cri-resmgr/cri-resource-manager.service | sudo tee /etc/systemd/system/cri-resource-manager.service
+
+sudo sed -i '/Requires=/d' /etc/systemd/system/cri-resource-manager.service
+sudo systemctl daemon-reload
+sudo systemctl restart cri-resource-manager.service
+sudo systemctl enable cri-resource-manager.service
+
diff --git a/clr-k8s-examples/10-cri-resource-manager/setup.sh b/clr-k8s-examples/10-cri-resource-manager/setup.sh
@@ -0,0 +1,24 @@
+#!/bin/bash
+
+set -o errexit
+set -o nounset
+
+CRI_RESMGR_SOCKET="/var/run/cri-resmgr/cri-resmgr.sock"
+KUBEADM_FLAGS="/var/lib/kubelet/kubeadm-flags.env"
+
+if sudo test -S "$CRI_RESMGR_SOCKET" ; then
+	sudo mkdir -p /etc/systemd/system/kubelet.service.d/
+	cat <<EOF | sudo tee /etc/systemd/system/kubelet.service.d/99-cri-resource-manager.conf
+[Service]
+Environment=KUBELET_EXTRA_ARGS=
+Environment=KUBELET_EXTRA_ARGS="--container-runtime remote --container-runtime-endpoint unix://${CRI_RESMGR_SOCKET}"
+EOF
+
+	if sudo test -f "$KUBEADM_FLAGS" ; then
+	    sudo mv $KUBEADM_FLAGS $KUBEADM_FLAGS.bkp
+	fi
+
+	sudo systemctl daemon-reload
+	sudo systemctl restart cri-resource-manager
+	sudo systemctl restart kubelet
+fi
diff --git a/clr-k8s-examples/README.md b/clr-k8s-examples/README.md
@@ -146,6 +146,9 @@ kubectl -n monitoring port-forward svc/grafana 3000
 Grafana is available at this URL http://localhost:3000 . Default credentials are
 `admin/admin`. Upon entering you will be asked to chose a new password.
 
+### CRI Resource Manager
+Go to [`10-cri-resource-manager`](./10-cri-resource-manager).
+
 ## Cleaning up the cluster (Hard reset to a clean state)
 
 Run `reset_stack.sh` on all the nodes