Merge pull request #101 from madflojo/develop

2017.06 Release

Author: madflojo

CONTRIBUTING.md

@@ -10,7 +10,7 @@ Before starting to contribute to Automatron please review and accept our [Contri
 
 Contributing to the Core platform and contributing Plugins have two different guidelines and requirements. The below will explain some basic concepts of how to contribute different functionality.
 
-### Core
+### Contributing to Core
 
 Automatron follows a pluggable architecture with the majority of features being provided by plugins located within the `plugins/` directory. This allows us to keep the Core framework minimal and simple.
 
@@ -23,10 +23,12 @@ At this time there are 4 primary core components of Automatron.
 
 These components are written in Python and as such should follow basic Python development practices.
 
-### Plugins
+### Contributing Plugins
 
 Where the Automatron Core provides a monitoring and actioning framework, the functional features are all provided by Automatron Plugins. Plugins, are the fastest way to add features to Automatron. As such it is suggested that new contributors start by adding a plugin before adding core functionality.
 
+Plugin contributions have been migrated to the [Automatron Plugins](https://github.com/Automatron-Plugins) project. Pull Requests for existing plugins should be opened against the appropriate repository in that project. Requests for new plugin repositories can be made via [Gitter](https://gitter.im/madflojo/automatron).
+
 #### Executable Plugins
 
 At this time there are 6 types of Plugins.
@@ -39,7 +41,7 @@ At this time there are 6 types of Plugins.
   * `vetting` - Executables used to identify `facts` for discovered monitoring targets.
 
 While `datastores`, `logging`, and `discovery` plugins are Python modules; `actions`, `checks` and `vetting` are simply executables.
-Python is the preferred approach however, these plugins may also be written in Perl or BASH. While other languages are accepted it is important to ensure capabilities are available across as many platforms as possible. When writing plugins do consider the availability of functionality on generic systems.
+These plugins may be written in Python, Perl, BASH or any language that results in a standalone executable.
 
 ## Contribution Workflow & Requirements
 
@@ -58,17 +60,15 @@ When opening a Pull Request for a bug fix, if the fix is for the current release
 
 If the bug fix should also be incorporated with the `develop` branch a second Pull Request should be opened to the `develop` branch.
 
-### Tests are required for Core and some Plugins
+### Tests are required for Core
 
 Any Pull Requests for the Automatron core code base should include applicable `unit`, `integration` and `functional` tests. Automatron uses Coveralls to ensure code coverage does not decrease with each Pull Request.
 
-While not strictly enforced, plugins should also include tests where applicable. In some cases it may not be possible to provide `unit` or `integration` tests for plugins. In these cases it is recommended to create `functional` tests.
-
 ### Documentation is required
 
-Documentation of new functionality is important to increase the adoption of Automatron. As such, you may be asked to provide documentation for new functionality created by your Pull Request. This is especially true for new plugins being submitted as plugins must be documented in order for users to adopt the plugin.
+Documentation of new functionality is important to increase the adoption of Automatron. As such, you may be asked to provide documentation for new functionality created by your Pull Request. This is especially true for major feature changes.
 
-Documentation is just as important as new functionality, as such documentation based pull requests are encouraged. For idea's on current gaps please reference our [documentation board](https://github.com/madflojo/automatron/projects/1).
+Documentation is just as important as new functionality, as such documentation based pull requests are encouraged.
 
 ## Developer environment
 

config/config.yml.example

@@ -35,29 +35,9 @@ discovery:
     webping:
       ip: 0.0.0.0
       port: 8000
-#    nmap:
-#      # NMAP Scanning for new hosts
-#      target: 10.0.0.1/8
-#      flags: -sP
-#      interval: 40
-#    digitalocean:
-#      # Query DO's API
-#      url: https://api.digitalocean.com/v2
-#      api_key: example
-#      interval: 60
-#    aws:
-#      # Query AWS' API
-#      aws_access_key_id: example
-#      aws_secret_access_key: example
-#      interval: 60
-#      filter:
-#        - PublicIpAddress
-#        - PrivateIpAddress
-#    linode:
-#      # Query Linode's API
-#      url: https://api.linode.com
-#      api_key: example
-#      interval: 60
+#    roster:
+#      hosts:
+#        - 10.0.0.1
 
 ## Datastore
 datastore:

config/runbooks/examples/cpu/idle.yml

@@ -0,0 +1,29 @@
+name: Check CPU Idle
+schedule:
+  seconds: "*/30"
+checks:
+  cpu_idle:
+    execute_from: target
+    type: cmd
+    cmd: test $(sar -u 1 1 | tail -n 1 | awk '{print $8}' | cut -d. -f1) -gt 10
+actions:
+  logger:
+    execute_from: target
+    type: cmd
+    cmd: echo "CPU Idle at critical threshold: `sar -u 1 1 | tail -n 1 | awk '{print $8}' | cut -d. -f1`" | logger -t automatron
+    trigger: 0
+    frequency: 120
+    call_on:
+      - CRITICAL
+      - WARNING
+      - UNKNOWN
+  reboot:
+    execute_from: target
+    type: cmd
+    cmd: reboot
+    trigger: 20
+    frequency: 900
+    call_on:
+      - WARNING
+      - CRITICAL
+      - UNKNOWN

config/runbooks/examples/disk_free/init.yml

@@ -1,5 +1,5 @@
 name: Clean up dangling images if they are found
-schedule: "*/2 * * * *"
+schedule: "0 */2 * * *"
 checks:
   find_danglers:
     execute_from: ontarget

config/runbooks/examples/docker/clear_dangling_images.yml

@@ -1,5 +1,5 @@
 name: Verify Docker is running
-schedule: "* * * * *"
+schedule: "*/2 * * * *"
 checks:
   docker_running:
     execute_from: ontarget

config/runbooks/examples/docker/init.yml

@@ -39,7 +39,7 @@ If the `gateway` setting is left as `False` Automatron will login to each host d
 
 By default, Automatron will listen on port `8000` for any HTTP requests. When an HTTP request is made to Automatron the IP will be captured and that server will then be identified as a monitoring target.
 
-There are several plugins that enable other methods for host discovery, in this section we will enable the `nmap` discovery plugin. This configuration is within the `discovery` section of the `config.yml` file.
+There are several plugins that enable other methods for host discovery, in this section we will also enable the `roster` discovery plugin. This configuration is within the `discovery` section of the `config.yml` file.
 
 ```yaml+jinja
 discovery:
@@ -52,7 +52,7 @@ discovery:
       port: 8000
 ```
 
-To enable the `nmap` plugin we simply need to append the `nmap` configuration within the `plugins` key.
+To enable the `roster` plugin we simply need to append the `roster` configuration within the `plugins` key.
 
 ```yaml+jinja
 discovery:
@@ -64,12 +64,11 @@ discovery:
       ip: 0.0.0.0
       port: 8000
     # NMAP Scanning
-    nmap:
-      target: 10.0.0.1/8
-      flags: -sP
-      interval: 40
+    roster:
+      hosts:
+        - 10.0.0.1
 ```
 
-Each plugin has unique configuration details, the specifics of these plugins can be found in the [plugin](plugins/index.md) documentation.
+Each plugin has unique configuration details, additional discovery plugins can be found in the [Automatron Plugins](https://github.com/Automatron-Plugins) project.
 
 At this point Automatron has been configured. We can now move on to creating our own [Runbooks](runbooks/index.md).

config/runbooks/examples/nginx/init.yml

@@ -1,4 +1,4 @@
-Auomatron leverages the power of [Jinja2](http://jinja.pocoo.org/docs/2.9/), a popular Python based templating language to enhance how runbooks can be used. The below example is a runbook that leverages Jinja2.
+Automatron leverages the power of [Jinja2](http://jinja.pocoo.org/docs/2.9/), a popular Python based templating language to enhance how runbooks can be used. The below example is a runbook that leverages Jinja2.
 
 ```yaml+jinja
 name: Check NGINX
@@ -44,7 +44,7 @@ actions:
 
 The above runbook leverages both Jinja2 and Automatron's internal **Facts**. Facts are variables that Automatron has collected during the Vetting process of each monitored system.
 
-When Automatron discovers a new host it executes [Vetting Plugins](plugins/#Vetting) on the host. Some plugins are executed remotely, others are executed on the host itself. These plugins return information unique to each host.
+When Automatron discovers a new host it executes Vetting Plugins on the host. Some plugins are executed remotely, others are executed on the host itself. These plugins return information unique to each host.
 
 An example of the type of information can be seen in the `ontarget/system_info.py` vetting plugin. This plugin creates facts for OS Distribution, Hostname, Kernel version and Network Information.
 

docs/configure.md

@@ -0,0 +1,74 @@
+Actions can be as simple as a one-liner shell command or as complicated as a 1,000 line Python script. With Automatron's action plugins both of these scenarios can be covered.
+
+Automatron action plugins are simply executables that are copied and executed on the specified target. There is no special language or coding syntax required other than the fact that the script should be executable without specifying the language.
+
+## A simple example
+
+To understand this further let's write a simple plugin that finds files that require a `dos2unix` conversion and converts them. Before getting into our script, let's start with a runbook for this example.
+
+```yaml+jinja
+name: Check if application has started
+schedule:
+  second: "*/30"
+checks:
+  port_443_is_up:
+    execute_from: target
+    type: plugin
+    plugin: network/tcp_connect.py
+    args: --host=localhost --port 443
+actions:
+  dos2unix_files:
+    execute_from: target
+    trigger: 0
+    frequency: 900
+    call_on:
+      - WARNING
+      - CRITICAL
+      - UNKNOWN
+    type: plugin
+    plugin: custom/dos2unix.sh
+    args: /application/path
+```
+
+The above runbook will verify if the application is listening on port 443 and if not execute the `dos2unix_files` action. This action is a plugin action which will copy the `plugins/actions/custom/dos2unix.sh` script to the target server and execute it with the argument of `/application/path`. To explain this better, the below command represents how this plugin would be executed on the target server.
+
+```sh
+$ custom/dos2unix.sh /application/path
+```
+
+This script itself can be very simple and only needs to accept the arguments passed.
+
+### Sample code
+
+The below script is a sample script that will perform a `dos2unix` conversion on any files within the specified path.
+
+```bash
+#!/bin/bash
+
+if [ -z $1 ]
+then
+  echo "Usage: $0 /path"
+  exit 1
+fi
+
+BEFORE=$(find $1 -type f -exec file {} \; | grep -c "with CRLF line terminators")
+
+for FILE in `find $1 -type f -exec file {} \; | grep "with CRLF line terminators" | cut -d: -f1`
+do
+  dos2unix $FILE
+done
+
+AFTER=$(find $1 -type f -exec file {} \; | grep -c "with CRLF line terminators")
+
+if [ $AFTER -lt $BEFORE ]
+then
+  exit 0
+else
+  exit 1
+fi
+```
+
+The exit codes used by the action plugin will be used by Automatron to determine successful action execution or a failed action execution. A script that is successful should exit with a code of `0`. Any other exit code will show as a failed action execution.
+
+!!! info
+    Automatron action plugins are designed to be as simple as possible to allow users to use their existing scripts along with Automatron's automated action execution. Aside from appropriate exit code usage there is no coding required to use an existing script or utility.