NoMethodError thrown in logstash/pipeline.rb using couchdb_changes on Alpha 5

[jarederaj]

This problem seems to be stemming from /logstash-input-couchdb_changes-3.0.2/lib/logstash/inputs/couchdb_changes.rb:144

• Version: ELK Alpha 5 release 5
• Plugins: coucdb_changes, x-pack
• Operating System: Ubuntu 16.04
• Config File:

input { couchdb_changes { db => "nt-jpad" } } output { file { path => "/home/jared/testing.txt" } }

• Sample Data:
  { "_id": "user-", "_rev": "1-d6788b98a9066b0ac439994807ed310a", "type": "user", "username": null, "created": 1467227196, "domains": { "chrome": { "created": 1467227196 } } }
• Steps to Reproduce:
  For Ubuntu 16.04
• Install Alpha 5.0 release 5 ELK stack
• Install x-pack plugin from elasticsearch
• Install x-pack plugin for kibana
• Install couchdb_changes for logstash
• implement any configuration for couchdb_changes and place in /etc/logstash/conf.d/yourconf.conf
• restart logstash
• Observe your log output for something like:

{:timestamp=>"2016-08-18T17:25:07.107000-0600",
:message=>"A plugin had an unrecoverable error. Will restart this plugin.
Plugin: <LogStash::Inputs::CouchDBChanges db=>"nt-jpad",
enable_metric=>true,
codec=><LogStash::Codecs::Plain enable_metric=>true,
charset=>"UTF-8">,
host=>"localhost",
port=>5984,
secure=>false,
password=>,
heartbeat=>1000,
keep_revision=>false,
ignore_attachments=>true,
always_reconnect=>true,
reconnect_delay=>10>
Error: undefined method [] for 2016-08-18T23:25:07.096Z %{host} %{message}:LogStash::Event
Exception: NoMethodError
Stack: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-couchdb_changes-3.0.2/lib/logstash/inputs/couchdb_changes.rb:167:in run' org/jruby/RubyArray.java:1613:ineach'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-couchdb_changes-3.0.2/lib/logstash/inputs/couchdb_changes.rb:154:in run' /usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/protocol.rb:395:incall_block'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/protocol.rb:386:in <<' /usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/protocol.rb:94:inread'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:2785:in read_chunked' /usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:2759:inread_body_0'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:2719:in read_body' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-couchdb_changes-3.0.2/lib/logstash/inputs/couchdb_changes.rb:153:inrun'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:1331:in transport_request' /usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:2680:inreading_body'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:1330:in transport_request' org/jruby/RubyKernel.java:1242:incatch'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:1325:in transport_request' /usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:1302:inrequest'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-couchdb_changes-3.0.2/lib/logstash/inputs/couchdb_changes.rb:148:in run' /usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:746:instart'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:557:in 'start'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-couchdb_changes-3.0.2/lib/logstash/inputs/couchdb_changes.rb:144:in run'
/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:323:in inputworker' /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:317:instart_input'",
:level=>:error,
:file=>"logstash/pipeline.rb",
:line=>"334",
:method=>"inputworker"}`

[jarederaj]

I'm still seeing this issue when I install using docker. Here's the debug output there:

Connecting to CouchDB _changes stream at: {:host=>"172.17.0.2", :port=>"5984", :db=>"test", :level=>:info, :file=>"logstash/inputs/couchdb_changes.rb", :line=>"139", :method=>"run"}
Using service uri : {:uri=>#<URI::HTTP:0x6af04cef URL:http://172.17.0.2:5984/test/_changes?feed=continuous&include_docs=true&since=0&heartbeat=1000>, :level=>:info, :file=>"logstash/inputs/couchdb_changes.rb", :line=>"141", :method=>"run"}
event {:event=>{"@timestamp"=>2016-08-29T19:34:13.307Z, "doc_as_upsert"=>true, "@metadata"=>{"action"=>"update", "_id"=>"1723902a398922503d6ba13e3d000613", "seq"=>2}, "@version"=>"1", "doc"=>{"testing"=>"type", "type"=>"testing"}}, :level=>:debug, :file=>"logstash/inputs/couchdb_changes.rb", :line=>"223", :method=>"build_event"}
event {:event=>{"@timestamp"=>2016-08-29T19:34:13.307Z, "doc_as_upsert"=>true, "@metadata"=>{"action"=>"update", "_id"=>"1723902a398922503d6ba13e3d000613", "seq"=>2}, "@version"=>"1", "doc"=>{"testing"=>"type", "type"=>"testing"}}, :level=>:debug, :file=>"logstash/inputs/couchdb_changes.rb", :line=>"164", :method=>"run"}
A plugin had an unrecoverable error. Will restart this plugin.
  Plugin: <LogStash::Inputs::CouchDBChanges db=>"test", port=>5984, host=>"172.17.0.2", enable_metric=>true, codec=><LogStash::Codecs::Plain enable_metric=>true, charset=>"UTF-8">, secure=>false, password=><password>, heartbeat=>1000, keep_revision=>false, ignore_attachments=>true, always_reconnect=>true, reconnect_delay=>10>
  Error: undefined method `[]' for 2016-08-29T19:34:13.307Z %{host} %{message}:LogStash::Event
  Exception: NoMethodError
  Stack: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-couchdb_changes-3.0.2/lib/logstash/inputs/couchdb_changes.rb:167:in `run'
org/jruby/RubyArray.java:1613:in `each'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-couchdb_changes-3.0.2/lib/logstash/inputs/couchdb_changes.rb:154:in `run'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/protocol.rb:395:in `call_block'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/protocol.rb:386:in `<<'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/protocol.rb:94:in `read'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:2785:in `read_chunked'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:2759:in `read_body_0'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:2719:in `read_body'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-couchdb_changes-3.0.2/lib/logstash/inputs/couchdb_changes.rb:153:in `run'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:1331:in `transport_request'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:2680:in `reading_body'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:1330:in `transport_request'
org/jruby/RubyKernel.java:1242:in `catch'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:1325:in `transport_request'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:1302:in `request'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-couchdb_changes-3.0.2/lib/logstash/inputs/couchdb_changes.rb:148:in `run'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:746:in `start'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:557:in `start'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-couchdb_changes-3.0.2/lib/logstash/inputs/couchdb_changes.rb:144:in `run'
/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:323:in `inputworker'
/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:317:in `start_input' {:level=>:error, :file=>"logstash/pipeline.rb", :line=>"334", :method=>"inputworker"}
Collector: Sending snapshot to observers {:created_at=>2016-08-29 19:34:13 +0000, :level=>:debug, :file=>"logstash/instrument/collector.rb", :line=>"104", :method=>"publish_snapshot"}
filter received {:event=>{"@timestamp"=>2016-08-29T19:34:13.307Z, "doc_as_upsert"=>true, "@version"=>"1", "doc"=>{"testing"=>"type", "type"=>"testing"}}, :level=>:debug, :file=>"(eval)", :line=>"45", :method=>"filter_func"}
output received {:event=>{"@timestamp"=>2016-08-29T19:34:13.307Z, "doc_as_upsert"=>true, "@version"=>"1", "doc"=>{"testing"=>"type", "type"=>"testing"}}, :level=>:debug, :file=>"(eval)", :line=>"52", :method=>"output_func"}
{
       "@timestamp" => 2016-08-29T19:34:13.307Z,
    "doc_as_upsert" => true,
        "@metadata" => {
        "action" => "update",
           "_id" => "1723902a398922503d6ba13e3d000613",
           "seq" => 2
    },
         "@version" => "1",
              "doc" => {
        "testing" => "type",
           "type" => "testing"
    }
}