SPPmon as a Container

SPPMon as a Container

SPPMon + Docker

Overview

The SPPMon program can be run within a container. The project includes a Dockerfile which builds a container from a Python 3.8 base and automatically pulls in the dependent Python modules and SPPMon code. This document explains how to build the SPPMon container, as well as set up a fully functioning SPPMon environment where Grafana and InfluxDB are also running within containers. Container persistent volumes are used to maintain the data used by Grafana and InfluxDB outside of the container images. The configuration files for all three containers are maintained under the /home file system of the host system.

Overview of Docker architecture

1. Install a host operating system

Testing of the containerized SPPMon solution was performed using CentOS 7.8. Other operating systems have not been tested but will certainly work.

1.1 Install CentOS 7.8

Create a virtual machine with 4 vcpu, 8 GB RAM, 1 x 50GB hdisk, and 1 x 100GB hdisk
Install the operating system on the 50GB disk (auto partitioning is acceptable)
For features to install, select server w/ GUI
Setup the network configuration in the installer
Set the root password, and create a non-root user sppmon with a secure password.
Following commands will be create everything for the user who executes them, so it is possible to use a different user than sppmon.

It is recommended to only use one user besides root to not mix permissions up.

1.2 Update the operating system and required software

Login as the sppmon user
su - root
Give the users root permissions:Create /etc/sudoers.d/${USER} containing the following one line: echo "${USER} ALL=(ALL) NOPASSWD: ALL" | sudo tee /etc/sudoers.d/${USER}
exit # back to sppmon
install docker
sudo yum update -y
sudo yum install docker -y

1.3 As the sppmon user continue creating a file system for persistent container storage

sudo pvcreate /dev/sdb
sudo vgcreate sppmonvg /dev/sdb
sudo lvcreate -a y -n sppmon -l 100%FREE sppmonvg
sudo mkfs -t xfs /dev/mapper/sppmonvg-sppmon
sudo mkdir /var/lib/docker/volumes
echo "/dev/mapper/sppmonvg-sppmon /var/lib/docker/volumes xfs defaults 0 0" | sudo tee -a /etc/fstab

1.4 Finish the docker setup and allow the sppmon user to manage containers

sudo groupadd docker
sudo usermod -aG docker $USER
su - ${USER} # Log back in so the new group assignments take effect.
sudo systemctl start docker
docker run hello-world # this should work as non-root
sudo yum install git -y
sudo reboot

1.5 Open firewall ports 8086 and 3000 for the Influxdb and Grafana containers

sudo firewall-cmd --add-port=8086/tcp --permanent
sudo firewall-cmd --add-port=3000/tcp --permanent
sudo firewall-cmd --reload
sudo systemctl restart docker

2. Setup all required Containers

2.1 Setup the SPPMon Container

2.1.1 Get the latest SPPMon code

Please refer to getting the SPPMon source code to get the latest code.

For copy-paste:

cd /home/sppmon
git clone https://github.com/IBM/spectrum-protect-sppmon.git

2.1.2 Build the SPPMon container (based on the Dockerfile in the SPPMon project)

cd /home/sppmon/spectrum-protect-sppmon/python
docker build -t sppmon .
docker image ls

Output:

 docker image ls
REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
sppmon                  latest              f40cb1362714        24 seconds ago      912 MB
docker.io/python        3.8                 b716d5a96fde        2 weeks ago         884 MB
docker.io/hello-world   latest              d1165f221234        4 months ago        13.3 kB

Test the SPPMon container. The following command should output the SPPMon --help:
docker run --rm sppmon

2.2 Setup the InfluxDB container

2.2.1 Setup an InfluxDB container using a persistent storage volume

docker pull influxdb:1.8.6
docker volume create influx-vol
mkdir /home/${USER}/influxconf
docker run --rm influxdb:1.8.6 influxd config > /home/${USER}/influxconf/influxdb.conf
Update the Influx-Config according to the steps of the regular setup. They are not included here to maintain consistency if changes are required.
You must include all changes to the config file. This includes turning authentication off.
vi /home/${USER}/influxconf/influxdb.conf

Run the influx DB container in detached mode. The config file and persistent data volume are attached to the expected location inside the container docker run -d -p 8086:8086 -v /home/${USER}/influxconf/influxdb.conf:/etc/${USER}/influxdb.conf:ro,Z -v influx-vol:/var/lib/influxdb --name influxdb influxdb:1.8.6 -config /etc/${USER}/influxdb.conf

2.2.2 Configure the InfluxDB

It is required to use a different password than shown here!

Create the initial admin user: curl -XPOST "http://localhost:8086/query" --data-urlencode "q=CREATE USER influxAdmin WITH PASSWORD 'NOT_A_SAFE_PASSWORD' WITH ALL PRIVILEGES"

Result: {"results":[{"statement_id":0}]}
Start a shell to the InfluxDB and issue the subsequent command within the shell to create a database, create users, and assign authority to the users.
You need to create a user GrafanaReader for Grafana to access the database without admin permissions.

It is required to use a different password than shown here!

docker exec -it influxdb influx -username influxAdmin -password NOT_A_SAFE_PASSWORD > CREATE USER GrafanaReader WITH PASSWORD 'NOT_A_SAFE_PASSWORD'

Note: For additional configuration options please check out the manual InfluxDB setup.

2.2.3 Turn Authentication back on

To create the users it was necessary to disable authentication. After creating both users it is required to re-enable authentification. Please make sure to have applied all required changes to the config file from the steps before and also apply the following change. Edit it via sudo nano /etc/influxdb/influxdb.conf or sudo vi /etc/influxdb/influxdb.conf.

Enable authentification

[http]

# Determines whether user authentication is enabled over HTTP/HTTPS.
auth-enabled = true

Remove the old InfluxDB image
2.1. docker stop influxdb
2.2. docker rm influxdb
2.3. docker run -d -p 8086:8086 -v /home/${USER}/influxconf/influxdb.conf:/etc/${USER}/influxdb.conf:ro,Z -v influx-vol:/var/lib/influxdb --name influxdb influxdb:1.8.6 -config /etc/${USER}/influxdb.conf
Test InfluxDB connection docker exec -it influxdb influx -ssl -unsafeSsl -host localhost -username influxAdmin -password NOT_A_SAFE_PASSWORD
show users

Please refer to the regular setup for the original edit of the config file again and change of the security settings.

2.2.4 Optional / Recommended: configure secure https for InfluxDB

docker stop influxdb
docker rm influxdb
openssl req -x509 -nodes -newkey rsa:4096 -keyout /home/${USER}/influxconf/influxdb-selfsigned.key -out /home/${USER}/influxconf/influxdb-selfsigned.crt
chmod 600 /home/${USER}/influxconf/influxdb-selfsigned.key
chmod 644 /home/${USER}/influxconf/influxdb-selfsigned.crt
vi /home/${USER}/influxconf/influxdb.conf # modify the lines below
[http]
https-enabled = true
https-certificate = "/etc/ssl/influxdb-selfsigned.crt"
https-private-key = "/etc/ssl/influxdb-selfsigned.key"

Start the docker container passing attaching the SSL certificates as a bind volume
docker run -d -p 8086:8086 -v /home/${USER}/influxconf/influxdb.conf:/etc/influxdb/influxdb.conf:ro,Z -v /home/${USER}/influxconf:/etc/ssl:ro,Z -v influx-vol:/var/lib/influxdb --name influxdb influxdb:1.8.6 -config /etc/influxdb/influxdb.conf
Test the connection via SSL
docker exec -it influxdb influx -ssl -unsafeSsl -host localhost -username influxAdmin -password NOT_A_SAFE_PASSWORD

Connected to https://localhost:8086 version 1.8.6
InfluxDB shell version: 1.8.6
> show databases
name: databases
name
----
_internal
sppmon
> quit

2.3 Setup the Grafana container

A Grafana container is set up using a persistent volume for Grafana metadata. This container is linked to the InfluxDB container so that it has network access.

Please refer to the official Grafana Documentation for an SSL-Setup and additional configuration options.
The manual setup also describes editing the config file.

2.3.1 Initial container setup

docker pull grafana/grafana
docker volume create grafana-vol
docker run -d -p 3000:3000 --link influxdb:influxdb --name=grafana -v grafana-vol:/var/lib/grafana grafana/grafana

Following commands do not need to be executed and are for information purposes only!

To connect a shell to the running detached container:
docker exec -ti grafana /bin/bash
To stop the grafana container:
docker stop grafana
To start the container:
docker start grafana

2.3.2 Verify the InfluxDB and Grafana containers are running

docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fca4eb208419 grafana/grafana "/run.sh" 51 seconds ago Up 49 seconds 0.0.0.0:3000->3000/tcp grafana
2a620c2c3e01 influxdb "/entrypoint.sh -c..." 14 minutes ago Up 14 minutes 0.0.0.0:8086->8086/tcp influxdb

3. SPPMon run configuration

3.1 Complete the SPPMon configuration

Note: The variable ${USER} does not need to be changed. Any CentOS system will automatically insert the current username.

A directory to hold log files is created under /home/${USER} (home user directory) under the local host storage.
Configuration files should be written into home/${USER}/spectrum-protect-sppmon/config_files/, with the SPP-Server name as filename: YOUR_SPP_SERVER_NAME.conf.
For further details, see Creating a sppmon .conf file

Create the .conf file for sppmon in /home/${USER}/spectrum-protect-sppmon/config_files/YOUR_SPP_SERVER_NAME.conf
cp /home/${USER}/spectrum-protect-sppmon/config_files/sppconnections_default.conf /home/${USER}/spectrum-protect-sppmon/config_files/YOUR_SPP_SERVER_NAME.conf
Edit the content of the config file vi /home/${USER}/spectrum-protect-sppmon/config_files/YOUR_SPP_SERVER_NAME.conf
Test running the sppmon container using the .conf file:
docker run --rm -v /home/${USER}/spectrum-protect-sppmon/config_files/YOUR_SPP_SERVER_NAME.conf:/usr/src/app/YOUR_SPP_SERVER_NAME.conf:ro,Z -v /home/${USER}/sppmonLogs:/root/sppmonLogs:Z sppmon python sppmon.py --cfg=YOUR_SPP_SERVER_NAME.conf --test

3.2 Setup Crontab

See recommended crontab configuration for more information.
You need to use the run command from above inside of crontab.
Substitute --test by the matching argument (--all, --daily, --hourly, and --constant)

3.3 Register the InfluxDB as a data source within Grafana

Follow the regular steps documented in the SPPMon wiki Configuring Grafana for sppmon. For the URL in the data source creation page, specify the following (note the hostname resolves to the correct IP address due to the --link of the Grafana and Influx containers):

This step only works if SPPMon has been executed at least once

using SSL- https://influxdb:8086
without SSL - http://influxdb:8086

3.4 SPPCheck Run command

To execute SPPCheck in a container is still experimental. It is required to add additional links and files to the Python container environment.

Link to Grafana: --link grafana:grafana
File access to the Sizer sheet: -v /tmp/Spectrum\ Protect\ Plus\ vSnap\ Sizer\ v1.9.xlsb:/usr/src/app/sizer.xlsb
Within the config file, in the grafana section, it is required to change the srv_address to grafana.

docker run --rm --link grafana:grafana -v /home/${USER}/spectrum-protect-sppmon/config_files/YOUR_SPP_SERVER_NAME.conf:/usr/src/app/YOUR_SPP_SERVER_NAME.conf:ro,Z -v /home/${USER}/sppmonLogs:/root/sppmonLogs:Z -v /tmp/_Spectrum\ Protect\ Plus\ vSnap\ Sizer\ v1.9.xlsb:/usr/src/app/sizer.xlsb sppmon python sppcheck.py --cfg=/usr/src/app/YOUR_SPP_SERVER_NAME.conf --startDate=2020-08-30 --predictYears=2 --sheet=/usr/src/app/sizer.xlsb --sizerVersion=v1.9 --pdfReport

SPPmon as a Container