Merge pull request #64 from madflojo/develop

MySQL and Network based health checks

docs/plugins/checks/mysql/available.md

@@ -0,0 +1,26 @@
+The `available` plugin is used to query MySQL's internal status system and determine whether the MySQL service is available or not.
+
+## Runbook Example
+
+The below is an example of using the `mysql/available` health check in a runbook.
+
+```yaml
+checks:
+  mysql_up:
+    execute_from: ontarget
+    type: plugin
+    plugin: mysql/available.py
+    args: --host=localhost --user=USERNAME --password=YOURPASSWORD
+```
+
+This plugin can be executed from either `ontarget` or `remote` depending on the MySQL service's configuration.
+
+### Required arguments
+
+The `mysql/available` plugin requires 3 arguments.
+
+```yaml
+args: -s <mysql host> -u <mysql user> -p <mysql password>
+```
+
+If the `show status` query is unsuccessful or does not find the key it is looking for the check will return a `CRITICAL` status. There is no `WARNING` or `UNKNOWN` status for this check.

docs/plugins/checks/mysql/status_metrics.md

@@ -0,0 +1,26 @@
+The `status_metrics` plugin is used to query MySQL's internal status system and alert when the defined `metric` exceeds the `warning` and `critical` thresholds.
+
+## Runbook Example
+
+The below is an example of using the `status_metrics` health check in a runbook.
+
+```yaml
+checks:
+  status_metrics:
+    execute_from: ontarget
+    type: plugin
+    plugin: mysql/status_metrics.py
+    args: --warn=20 --critical=10 --metric=slow_queries --host=localhost --user=USERNAME --password=YOURPASSWORD --type=greater
+```
+
+This plugin can be executed from either `ontarget` or `remote` depending on the MySQL service's configuration.
+
+### Required arguments
+
+The `mysql/status_metrics` plugin requires 7 arguments.
+
+```yaml
+args: -w <warning value> -c <critical value> -t {greater, lesser} -m <metric> -s <mysql host> -u <mysql user> -p <mysql password>
+```
+
+The `type` flag is used to define whether or not the alert is triggered when the `metric` value is "greater" or "lesser" than the values defined as `warn` and `critical`.

docs/plugins/checks/network/ping.md

@@ -0,0 +1,26 @@
+The `ping` plugin is used to identify if a `host` is online by sending an ICMP packet. This plugin uses `bash` and the `ping` command, which should work on most Unix and Linux systems.
+
+## Runbook Example
+
+The below is an example of using the `network/ping` health check in a runbook.
+
+```yaml
+checks:
+  host_up:
+    execute_from: remote
+    type: plugin
+    plugin: network/ping.sh
+    args: -i 10.0.0.1
+```
+
+This plugin can be executed from either `ontarget` or `remote` depending on the goal of the runbook.
+
+### Required arguments
+
+The `network/ping` plugin requires 1 argument.
+
+```yaml
+args: -i <ip or hostname> [-t <timeout value>]
+```
+
+The second value `-t` is an optional value, default is **3 seconds**.

docs/plugins/checks/network/tcp_connect.md

@@ -0,0 +1,26 @@
+The `tcp_connect` plugin is used to identify if a `host` or `ip` is listening on a specified `port`. This is a simple check that is either `OK` for successful connections or `CRITICAL` for unsuccessful connections.
+
+## Runbook Example
+
+The below is an example of using the `network/tcp_connect` health check in a runbook.
+
+```yaml
+checks:
+  mysql_up:
+    execute_from: ontarget
+    type: plugin
+    plugin: network/tcp_connect.py
+    args: --host=localhost --port 3306
+```
+
+This plugin can be executed from either `ontarget` or `remote` depending on the target being monitored.
+
+### Required arguments
+
+The `network/tcp_connect` plugin requires 2 arguments.
+
+```yaml
+args: -i <ip or hostname> -p <port> [-t <timeout value>]
+```
+
+The third value `-t` is an optional value, default is **5 seconds**.

mkdocs.yml

@@ -21,6 +21,12 @@ pages:
       - Web Ping: plugins/discovery/webping.md
       - Digital Ocean: plugins/discovery/digitalocean.md
     - Checks:
+      - Network:
+        - Ping: plugins/checks/network/ping.md
+        - TCP Connection: plugins/checks/network/tcp_connect.md
+      - MySQL:
+        - Available: plugins/checks/mysql/available.md
+        - Status Metrics: plugins/checks/mysql/status_metrics.md
       - Systems:
         - Disk Free: plugins/checks/systems/disk_free.md
         - Memory Free: plugins/checks/systems/mem_free.md

plugins/checks/mysql/available.py

@@ -0,0 +1,61 @@
+#!/usr/bin/env python
+
+import pymysql
+import sys
+import argparse
+
+def get_status(args):
+    ''' Pull the status values from MySQL '''
+    status = {}
+    try:
+        db = pymysql.connect(
+            host=args.host,
+            user=args.user,
+            password=args.password,
+            cursorclass=pymysql.cursors.DictCursor)
+    except: #pylint disable=broad-except
+        return False
+
+    try:
+        with db.cursor() as cursor:
+            # Get Status
+            cursor.execute("show status")
+            for result in cursor.fetchall():
+                try:
+                    status[result['Variable_name']] = int(result['Value'])
+                except: #pylint disable=broad-except
+                    pass
+    finally:
+        db.close()
+
+    if "Uptime" in status.keys():
+        return status['Uptime']
+    else:
+        return False
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "-s", "--host",
+        help="MySQL Host", required=True)
+    parser.add_argument(
+        "-u", "--user",
+        help="MySQL User", required=True)
+    parser.add_argument(
+        "-p", "--password",
+        help="MySQL User Password", required=True)
+    args = parser.parse_args()
+
+
+    if get_status(args):
+        alert = "OK"
+        status = "UP"
+        exit_code = 0
+    else:
+        alert = "CRITICAL"
+        status = "DOWN"
+        exit_code = 2
+
+    print "MYSQL_AVAILABLE {0} {1}".format(alert, status)
+    sys.exit(exit_code)

plugins/checks/mysql/status_metrics.py

@@ -0,0 +1,88 @@
+#!/usr/bin/env python
+
+import pymysql
+import sys
+import argparse
+
+def get_status(args):
+    ''' Pull the status values from MySQL '''
+    status = {}
+    try:
+        db = pymysql.connect(
+            host=args.host,
+            user=args.user,
+            password=args.password,
+            cursorclass=pymysql.cursors.DictCursor)
+    except: #pylint disable=broad-except
+        return False
+
+    try:
+        with db.cursor() as cursor:
+            # Get Status
+            cursor.execute("show status")
+            for result in cursor.fetchall():
+                try:
+                    status[result['Variable_name'].lower()] = int(result['Value'])
+                except: #pylint disable=broad-except
+                    pass
+    finally:
+        db.close()
+
+    return status
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "-w", "--warn", type=float,
+        help="Warning threshold value", required=True)
+    parser.add_argument(
+        "-c", "--critical", type=float,
+        help="Critical threshold value", required=True)
+    parser.add_argument(
+        "-m", "--metric",
+        help="Metric to validate (i.e. slow-queries)", required=True)
+    parser.add_argument(
+        "-s", "--host",
+        help="MySQL Host", required=True)
+    parser.add_argument(
+        "-u", "--user",
+        help="MySQL User", required=True)
+    parser.add_argument(
+        "-p", "--password",
+        help="MySQL User Password", required=True)
+    parser.add_argument(
+        "-t", "--type",
+        help="Alert if metric's value is greater or lesser than thresholds",
+        choices=('greater', 'lesser'),
+        required=True)
+    args = parser.parse_args()
+    metric = args.metric.lower()
+
+
+    # Get status
+    status = get_status(args)
+    alert = "OK"
+    exit_code = 0
+
+    if status and metric in status.keys():
+        if args.type == "greater":
+            if args.warn < status[metric]:
+                alert = "WARNING"
+                exit_code = 1
+            if args.critical < status[metric]:
+                alert = "CRITICAL"
+                exit_code = 2
+        else:
+            if args.warn > status[metric]:
+                alert = "WARNING"
+                exit_code = 1
+            if args.critical > status[metric]:
+                alert = "CRITICAL"
+                exit_code = 2
+    else:
+        print "Could not pull stats from MySQL"
+        sys.exit(3)
+
+    print "MYSQL_STATUS_HEALTH {0} {1} {2}".format(alert, metric, status[metric])
+    sys.exit(exit_code)

plugins/checks/network/ping.sh

@@ -0,0 +1,45 @@
+#!/usr/bin/env bash
+
+usage() {
+  echo 'Usage: '`basename $0` '-i <host> [-t <timeout>]'
+  exit 1
+}
+
+timeout=3 #Default value
+
+# Parse command line arguments
+while getopts ":i:t:" opts
+do
+  case $opts in
+    i)
+      host=${OPTARG}
+      ;;
+    t)
+      timeout=${OPTARG}
+      ;;
+    *)
+      echo "Unknown argument:" ${OPTARG}
+      usage
+      ;;
+  esac
+done
+
+if [ -z $host ]
+then
+  usage
+fi
+
+ping $host -c 1 -w $timeout 2>&1 > /dev/null
+if [ $? -eq 0 ]
+then
+  alert="OK"
+  status="UP"
+  exit=0
+else
+  alert="CRITICAL"
+  status="DOWN"
+  exit=2
+fi
+
+echo "PING $alert $status"
+exit $exit

plugins/checks/network/tcp_connect.py

@@ -0,0 +1,45 @@
+#!/usr/bin/env python
+
+import socket
+import argparse
+import sys
+
+def check_connection(args):
+    ''' Open TCP Connection and Return if successful '''
+    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+    s.settimeout(args.timeout)
+    try:
+        return not bool(s.connect_ex((args.host, args.port)))
+    except socket.error:
+        return False
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "-i", "--host",
+        help="IP or Host Address", required=True)
+    parser.add_argument(
+        "-p", "--port",
+        help="Port",
+        type=int,
+        required=True)
+    parser.add_argument(
+        "-t", "--timeout",
+        help="Timeout in seconds",
+        type=int,
+        required=False,
+        default=5
+    )
+    args = parser.parse_args()
+
+    if check_connection(args):
+        alert = "OK"
+        status = "UP"
+        exit_code = 0
+    else:
+        alert = "CRITICAL"
+        status = "DOWN"
+        exit_code = 2
+
+    print "TCP_CONNECT {0} {1}".format(alert, status)
+    sys.exit(exit_code)