Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Delay #378

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ Plugins
* [AlertOps](plugins/alertops)
* [AMQP](plugins/amqp)
* [Cachet](plugins/cachet)
* [Delay](plugins/delay)
* [DingTalk](plugins/dingtalk)
* [Enhance](plugins/enhance)
* [Forward](plugins/forward)
Expand Down
62 changes: 62 additions & 0 deletions plugins/delay/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Delay filter plugin

[Shelve](https://docs.alerta.io/en/latest/cli.html#shelve-shelve-alerts) alerts to prevent notifications within a set timout.
The delay plugin uses [Alerta Filter](https://docs.alerta.io/en/latest/server.html#filters).

A received alert that matches any filters will be shelved. The matching is the same as for [Blackout Periods](https://docs.alerta.io/en/latest/server.html#blackout-periods).
The alert will remain shelved for the duration of the timeout set in filter, config or in the plugin it self.
If the alert hasn't cleared within the timout, [housekeeper](https://docs.alerta.io/en/latest/cli.html#housekeeping-expired-and-clears-old-alerts) will pick up the alert and unshelve it.


## Installation

Clone the GitHub repo and run:

$ python setup.py install

Or, to install remotely from GitHub run:

$ pip install git+https://github.com/alerta/alerta-contrib.git#subdirectory=plugins/delay

Note: If Alerta is installed in a python virtual environment then plugins need to be installed into the same environment for Alerta to dynamically discover them.


## Configuration

`DELAY_TIMEOUT` - Integer. The default timeout for delay filters. Plugin defaults to `120` if not set.
`DELAY_MATH` - String. 'max' or 'min'. What timeout to choose. `'max'` will take the highest timeout of the matching filters. Plugin defaults to the first match if not set.
`DELAY_AUTOUNSHELVE` - Boolean. By default an re-opened alert will enter it's last status. If the configuration is set to `True`, alerts will be auto-unshelved when re-opened. This is for alerts where filters does not match. Plugin defaults to `False` if not set.

```python
PLUGINS = ['delay']
DELAY_TIMEOUT = 120
DELAY_MATH = 'max'
DELAY_AUTOUNSHELVE = True
```


## Parameters

Parameters to pass to filters are:
`type: "delay"` - To create a filter with the delay plugin.
`attributes: {"timeout": NNN}` - Timeout in seconds (integer). How long the alert is to be shelved.

See [Alerta API - Filters](https://docs.alerta.io/en/latest/api/reference.html#filters) for all inputs.


### Examples

```
curl -XPOST http://localhost:8080/filter \
-H 'Authorization: Key demo-key' \
-H 'Content-type: application/json' \
-d '{
"environment": "Production",
"service": ["example.com"],
"type": "delay",
"attributes": {
"timeout": 300
}
}'
```

80 changes: 80 additions & 0 deletions plugins/delay/alerta_delay.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
import logging

from alerta.app import alarm_model
from alerta.models.alarms.alerta import SEVERITY_MAP
from alerta.models.enums import Status, Action
from alerta.models.filter import Filter
from alerta.plugins import PluginBase

LOG = logging.getLogger('alerta.plugins')


class DelayHandler(PluginBase):

def pre_receive(self, alert, **kwargs):
return alert

def post_receive(self, alert, **kwargs):
DELAY_TIMEOUT = self.get_config('DELAY_TIMEOUT', default=120, type=int, **kwargs)
DELAY_MATH = self.get_config('DELAY_MATH', default=None, type=str, **kwargs)
DELAY_AUTOUNSHELVE = self.get_config('DELAY_AUTOUNSHELVE', default=False, type=bool, **kwargs)
CLEARED_LEVEL = SEVERITY_MAP[alarm_model.DEFAULT_NORMAL_SEVERITY]
TIMEOUTS = []

# If alert severity level matches DEFAULT_NORMAL_SEVERITY in SEVERITY_MAP
# then severity is cleared
# https://docs.alerta.io/configuration.html?highlight=#severity-settings
if SEVERITY_MAP[alert.severity] == CLEARED_LEVEL:
return alert

filters = [f.serialize for f in Filter.find_matching_filters(alert, "delay")]
if not filters:
# Override default behavior of alerta
if alert.status in Status.Shelved and DELAY_AUTOUNSHELVE:
alert = alert.from_action(Action.UNSHELVE, "Unshelved alert according to delay rules", self.get_config('ALERT_TIMEOUT'))
LOG.debug(f'Unshelved alert according to delay rules (id={alert.id}) (action={Action.UNSHELVE})')
return alert

if alert.status in Status.Shelved:
return alert

for f in filters:
if 'timeout' not in f['attributes']:
LOG.debug(f"No 'timeout' in filter (filter id={id})")
continue

timeout = f['attributes']['timeout']
try:
timeout = int(timeout)
except ValueError:
LOG.debug(f"Could not convert 'timeout' to an integer (filter id={id}) (timeout={timeout})")
continue

if timeout < 0:
LOG.debug(f"Invalid negative 'timeout' value (filter id={id}) (timeout={timeout})")
continue

TIMEOUTS.append(timeout)


if len(TIMEOUTS) > 0:
switcher = {
'max': max(TIMEOUTS),
'min': min(TIMEOUTS),
None: TIMEOUTS[0]
}

DELAY_TIMEOUT = switcher.get(DELAY_MATH)

alert = alert.from_action(Action.SHELVE, "Shelved alert according to delay rules", DELAY_TIMEOUT)
LOG.debug(f'Shelved alert according to delay rules (id={alert.id}) (timeout={alert.timeout}) (action={Action.SHELVE})')
return alert

def status_change(self, alert, status, text, **kwargs):
return

def take_action(self, alert, action, text, **kwargs):
raise NotImplementedError

def delete(self, alert, **kwargs) -> bool:
raise NotImplementedError
22 changes: 22 additions & 0 deletions plugins/delay/setup.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
from setuptools import setup, find_packages

version = '1.0.0'

setup(
name="alerta-delay",
version=version,
description='Alerta plugin for delay (autoshelve) to prevent notifications within a set timout.',
url='https://github.com/alerta/alerta-contrib',
license='MIT',
author='Terje Solend Nomeland, Are Schjetne, Øystein Middelthun',
author_email='[email protected], [email protected], [email protected]',
packages=find_packages(),
py_modules=['alerta_delay'],
include_package_data=True,
zip_safe=True,
entry_points={
'alerta.plugins': [
'delay = alerta_delay:DelayHandler'
]
}
)