msoak.1

.\" msoak.1 Copyright (C) 2019-2020 Jan-Piet Mens <jp@mens.de>
.\" ===
.TH msoak 1 "January 2020" "jpmens" "MQTT utilities"
.\"-----------------------------------------------------------
.SH NAME
msoak \- consume messages from arbitrary MQTT brokers and subscriptions
.\"-----------------------------------------------------------
.SH SYNOPSIS
.B msoak
[
.I -v
]
.I configuration
.\"-----------------------------------------------------------
.SH DESCRIPTION
.I msoak
is a utility with which to simultaneously subscribe to an arbitrary number of topics on any number of MQTT brokers and optionally modify or normalize received payloads before printing them out. This utility was created to back up to a central location messages received by a hand full of brokers; instead of launching (and having to monitor success of) a large number of
.IR mosquitto_sub (1)
programs,
.I msoak
took on the job.
.PP
.I msoak
uses asynchronous connects to the MQTT brokers so that it can handle situations in which a broker may temporarily be unavailable even when
.I msoak
initially starts, and
.I msoak
embeds a Lua interpreter for which you can create scripts which act on messages received to, for example, trigger actions.
.PP
.I msoak
can be used with, say,
.IR tinylog (8)
to create compressed archives of data.
.sp
.ft CW
msoak my.config | tinylog -k 30 -s 1077936128 -z archive
.ft
.PP

.\"-----------------------------------------------------------
.SH OPTIONS
.IP \fC-v\fR 0.5i
print version information and exit.
.\"-----------------------------------------------------------
.SH CONFIGURATION
.I msoak
reads the
.I configuration
file using
.IR libconfig (3)
utility functions. This is a short example:
.sp
.nf
.in 1i
.ft CW
# this is a comment

luascript = "example.lua"

servers = (
            { id = "local"
              host = "localhost"
              port = 1883
              fmt = "greet"
              showid = true
              showtopic = true
              topics = [
                 "tests/sponge/#",
                 "some/thing/else/+/+"
              ]
            },
            { id = "tmo"
              host = "test.mosquitto.org"
              qos = 0
              showid = true
              showtopic = true
              topics = [
                  "home/special",
                  "nothome/dont"
              ]
            }
        );
.ft
.in
.fi
.sp
The configuration consists of
.B global
settings and a list or array of
.BR servers ,
each with a number of settings.

.\"------------------------------
.SS global variables
.IP luascript 1i
the path to the Lua script file to use. If left unset, no Lua processing is performed.
.IP verbose 1i
whether or not to be verbose (default true)
.PP
.\"------------------------------
.SS server array

.IP id 1i
identifier for the connection, also shown on output. If
.I id
is unset we generate a numeric
.IR id.
.IP host 1i
host name or address of the MQTT broker. Defaults to localhost.
.IP port 1i
the TCP port number for the MQTT broker. Defaults to 1883.
.IP clientid 1i
MQTT clientId to use. The client identifier defaults to the string
.I msoak-hostname:basename(configfilename)
so that the program can run on multiple hosts with the same configuration.
.IP qos 1i
MQTT QoS (default 1)
.IP user 1i
username for the MQTT connection
.IP pass 1i
password for the MQTT connection; see
.I passenv
to omit from having to specify clear-text passwords in the
.I configuration
file.
.IP passenv 1i
name of environment variable (including $) from which to obtain
.IR pass .
.IP showretained 1i
consume retained messages; default true
.IP cacert 1i
path to PEM-encoded CA certificate chain for TLS connections to MQTT broker
.IP showid 1i
true or false, default true; whether to print
.I id
on output.
.IP showtopic 1i
true or false, default true; whether to print topic name on output.
.IP topics 1i
array of strings with topic branch names for
.I msoak
to subscribe to.
.IP fmt 1i
optional name of Lua formatting function (see below).
.PP
.\"------------------------------
.SS Formatting
By default, the received payload is printed to standard output, optionally
prefixed by the message topic (if
.I showtopic
is true), and it is preceeded by the connection identifier if
.I showid
is true.
.PP
If
.I fmt
is set, it contains a string with the name of a Lua function from the Lua
script specified in the
.I luascript
global. this function is used to format the payload
.I msoak
receives; the return value of the function replaces the original payload and is
printed out.
.PP
If
.I msoak
can decode the payload into JSON (i.e. the message begins with a brace
({) and the JSON can be decoded), it will invoke the Lua function to obtain output.
.PP
\fR
.nf
.ft CW
.in 1i
function init()
    if msoak.verbose then
        msoak.log(msoak.luascript .. " starting up")
    end
end

function greet(topic, _type, d)
    s = string.format("Hello %s -> now=%s", d.name,
        msoak.strftime("%TZ"))
    return s
end

function exit()
    msoak.log("Hasta la vista, baby!")
end
.fi
.ft
.in
.PP
The optional
.IR init ()
and
.IR exit ()
functions are invoked when
.I msoak
begins and ends respectively. The
.I greet
function is invoked for each message for which a server in the
.I configuration
file contains \fBfmt = "greet"\fR.
.\"------------------------------
.SS Lua functions
There are a few variables and functions
.I msoak
implements which are available to the Lua scripts you use.
.IP version 1i
returns the
.I msoak
version number as a string
.IP luascript 1i
returns the file name of the
.I luascript
global variable
.IP verbose 1i
is a boolean which indicates whether
.I msoak
is running in verbose mode
.IP msoak_log() 1i
accepts a string which is printed to
.I stderr
prefixed by "MSOAKLOG:".
.IP msoak_strftime() 1i
expects a
.I format
string and integer
.I seconds
and implements
.IR strtime (3)
for Lua with the specified format and seconds and returns the string result to Lua.
As a special case, if
.I seconds is less than one it uses current time.

.\"------------------------------
.SS JSON
When configured to use a Lua script file,
.I msoak
attempts to decode incoming JSON payloads and will pass the decoded JSON
elements to the configured
.I fmt
function as a table with these additional elements in it
.IP _conn_id 1i
the original connection
.I id
.IP _conn_host 1i
the hostname of the connection
.IP _conn_port 1i
the port number of the connection
.IP _conn_topic 1i
the MQTT topic on which the original payload message was received
.PP
Note that if no luascript was specified and the payload contains JSON it will be dumped as is to stdout.
.\"-----------------------------------------------------------
.SH ENVIRONMENT
Any number of environment variables may be used by
.I msoak
if specified in
.I passenv
settings with the configuration.
.\"-----------------------------------------------------------
.SH BUGS
What's with the strange name? Just as I started working on this program
I learned about
.IR sponge (1)
and loved the name. The rest is history.
.PP
Note that there are different versions of
.IR libconfig (3)
floating around which may have effects on the syntax permitted in
.IR msoak 's
.I configuration
file .
.\"-----------------------------------------------------------
.SH AUTHOR
Jan-Piet Mens, https://github.com/jpmens/msoak
.\"-----------------------------------------------------------
.SH SEE ALSO
.nh
.BR mosquitto_sub (1),
.BR mqttwarn ,
.BR sponge (1),
.BR strftime (1),
.BR tinylog (8)
.\" EOF msoak.1