Skip to content

ierceg/winston-elasticsearch

 
 

Repository files navigation

winston-elasticsearch

Version npmBuild StatusDependenciesCoverage Status

An elasticsearch transport for the winston logging toolkit.

Features

  • logstash compatible message structure.
  • Thus consumable with kibana.
  • Date pattern based index names.
  • Custom transformer function to transform logged data into a different message structure.

Compatibility

For Elasticsearch 5.0 and later, use the 0.5.x series. For earlier versions, use the 0.4.x series.

Unsupported / Todo

  • Querying.
  • Real buffering of messages in case of unavailable ES.

Installation

npm install --save winston winston-elasticsearch

Usage

var winston = require('winston');
var Elasticsearch = require('winston-elasticsearch');

var esTransportOpts = {
  level: 'info'
};
winston.add(winston.transports.Elasticsearch, esTransportOpts);

// - or -

var logger = new winston.Logger({
  transports: [
    new Elasticsearch(esTransportOpts)
  ]
});

Options

  • level [info] Messages logged with a severity greater or equal to the given one are logged to ES; others are discarded.
  • index [none] the index to be used. This option is mutually exclusive with indexPrefix.
  • indexPrefix [logs] the prefix to use to generate the index name according to the pattern <indexPrefix>-<indexSuffixPattern>.
  • indexSuffixPattern [YYYY.MM.DD] a Moment.js compatible date/ time pattern.
  • messageType [log] the type (path segment after the index path) under which the messages are stored under the index.
  • transformer [see below] a transformer function to transform logged data into a different message structure.
  • ensureMappingTemplate [true] If set to true, the given mappingTemplate is checked/ uploaded to ES when the module is sending the fist log message to make sure the log messages are mapped in a sensible manner.
  • mappingTemplate [see file index-template-mapping.json file] the mapping template to be ensured as parsed JSON.
  • flushInterval [2000] distance between bulk writes in ms.
  • client An elasticsearch client instance. If given, all following options are ignored.
  • clientOpts An object hash passed to the ES client. See its docs for supported options.
  • waitForActiveShards [1] Sets the number of shard copies that must be active before proceeding with the bulk operation.

Important

When changing the indexPrefix and/ or the transformer, make sure to provide a matching mappingTemplate.

Transformer

The transformer function allows to transform the log data structure as provided by winston into a sturcture more appropriate for indexing in ES.

The default transformer function's transformation is shwon below.

Input:

{
  "message": "Some message",
  "level": "info",
  "meta": {
    "method": "GET",
    "url": "/sitemap.xml",
    ...
    }
  }
}

Output:

{
  "@timestamp": "2015-09-30T05:09:08.282Z",
  "message": "Some message",
  "severity": "info",
  "fields": {
    "method": "GET",
    "url": "/sitemap.xml",
    ...
  }
}

The @timestamp is generated in the transformer. Note that in current logstash versions, the only "standard fields" are @timestamp and @version, anything else ist just free.

A custom trunsformer function can be provided in the options hash.

Events

  • error: in case of any error.

Example

An example assuming default settings.

Log Action

logger.info('Some message', <req meta data>);

Generated Message

The log message generated by this module has the following structure:

{
  "@timestamp": "2015-09-30T05:09:08.282Z",
  "message": "Some log message",
  "severity": "info",
  "fields": {
    "method": "GET",
    "url": "/sitemap.xml",
    "headers": {
      "host": "www.example.com",
      "user-agent": "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)",
      "accept": "*/*",
      "accept-encoding": "gzip,deflate",
      "from": "googlebot(at)googlebot.com",
      "if-modified-since": "Tue, 30 Sep 2015 11:34:56 GMT",
      "x-forwarded-for": "66.249.78.19"
    }
  }
}

Target Index

This message would be POSTed to the following endpoint:

http://localhost:9200/logs-2015.09.30/log/

So the default mapping uses an index pattern logs-*.

About

An elasticsearch transport for winston

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • JavaScript 100.0%