Grafana e mail alert notifications

e-mail notifications

Grafana includes an alerting system and provides the ability to push notifications to various notification receivers. The dashboard templates provided with SPPMON include pre-defined alerts for many common issues that are important to monitor. By default, these alerts will only be reported inside of the dashboard itself. This topic provides the steps necessary to enable e-mail notification of the alerts in addition to the summaries on the dashboard. For more information in the Grafana documentation, including details on the other alert channels that are available, see: https://grafana.com/docs/grafana/latest/alerting/notifications/ Setting up a notification channel via Slack is described here.

SPPMON e-mail alert setup instructions

Install the Grafana image rendering plug-in

This step is optional but recommended. The plugin allows images of the alerting graphs to be included in the e-mail body.

1. Install the Grafana image renderer plugin sudo grafana-cli plugins install grafana-image-renderer sudo yum install libXScrnSaver
2. Check for missing dependencies for the image-renderer
   • Find missing prerequisites, looking for anything reported as not found. cd /var/lib/grafana/plugins/grafana-image-renderer ldd chrome-linux/chrome | grep "not found"
   • Install any missing packages using yum, then repeat the above ldd command.

Setup and enable the Grafana SMTP feature

1. Stop the Grafana service systemctl stop grafana

2. Edit the smtp stanza in the Grafana configuration file to set parameters such as the host vi /etc/grafana/grafana.ini

   #################################### SMTP / Emailing ##########################
   [smtp]
   enabled = true
   host = relay.yourco.com:25
   ;user =
   `# If the password contains # or ; you have to wrap it with triple quotes. Ex """#password;"""`
   ;password =
   ;cert_file =
   ;key_file =
   skip_verify = true
   from_address = [email protected]
   from_name = """IBM Spectrum Protect Plus monitor"""
   # EHLO identity in SMTP dialog (defaults to instance_name)
   ;ehlo_identity = dashboard.example.com

3. Restart the Grafana service systemctl start grafana

Add an e-mail alerting channel

1. Click the bell icon within Grafana to open the Notification channels settings
2. Add a new notification channel of type e-mail

Enable the e-mail alert channel for specific alerts

Panels on the Grafana dashboard, which have alerts defined, can be identified by the heart icon. The icon precedes the panel title.

1. Edit a panel on the Grafana dashboard by select Edit from the drop-down list, which can be opened by clicking next to the panel title.
2. Click on the Alert tab of the panel edit page.
3. Under the Notifications section, choose the e-mail notification channel created in the earlier step as the "send to" location.

Customize alert thresholds

You are also able to adjust the conditions which trigger alerts from the Alert tab of a panel edit page. For example, you might want to adjust the minimum backup success rate which is allowed before an alert is triggered