secure option does not actually use https protocol

[m90]

We have recently migrated our SSL setup and are since then seeing this plugin crash our pipeline with the following error:

[INFO ] 2020-08-24 13:53:27.136 [[main]<couchdb_changes] couchdbchanges - Connecting to CouchDB _chan
ges stream at: {:host=>"couchdb.example.com", :port=>"443", :db=>"example"}                 
[INFO ] 2020-08-24 13:53:27.136 [[main]<couchdb_changes] couchdbchanges - Using service uri : {:uri=>
#<URI::HTTP http://couchdb.example.com:443/example/_changes?feed=continuous&include_docs=tru
e&since=xxxyyyzzz&heartbeat=1000>}
[ERROR] 2020-08-24 13:53:27.139 [[main]<couchdb_changes] pipeline - A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::CouchDBChanges password=><password>, port=>443, host=>"couchdb.example.com", id=>"example", secure=>true, db=>"example", username=>"example-elk", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_23b744f2-e76c-4719-8ff3-50f5a3c2d664", enable_metric=>true, charset=>"UTF-8">, heartbeat=>1000, keep_id=>false, keep_revision=>false, ignore_attachments=>true, always_reconnect=>true, reconnect_delay=>10>
  Error: Received fatal alert: handshake_failure
  Exception: OpenSSL::SSL::SSLError
  Stack: org/jruby/ext/openssl/SSLSocket.java:276:in `connect_nonblock'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:938:in `connect'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:868:in `do_start'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:857:in `start'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:585:in `start'
/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-couchdb_changes-3.1.6/lib/logstash/inputs/couchdb_changes.rb:157:in `run'
/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:409:in `inputworker'
/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:403:in `block in start_input'

which has me thinking the plugin is constructing a URL like http://couchdb.example.com:443 when I would expect it to use https://couchdb.example.com:443 (or no port at all) as we are using the secure option in our config:

couchdb_changes {
  id => 'example'
  db => 'example'
  host => "couchdb.example.com"
  secure => true
  port => 443
  username => "user"
  password => "pass"
}

Our current setup is behind Caddy which rejects http requests against port 443 with (I have to admit I have no idea how this worked when we were using nginx previously):

➜  ~ curl http://couchdb.example.com:443 
Client sent an HTTP request to an HTTPS server.

---

I looked at the code that constructs the URL and discovered something that makes me think this might be a bug. build_uri looks like this:

logstash-input-couchdb_changes/lib/logstash/inputs/couchdb_changes.rb

Lines 201 to 206 in 31d18d8

	private
	def build_uri
	options = {:feed => FEED, :include_docs => INCLUDEDOCS, :since => @sequence}
	options = options.merge(@timeout ? {:timeout => @timeout} : {:heartbeat => @heartbeat})
	URI::HTTP.build(:scheme => @scheme, :host => @host, :port => @port, :path => @path, :query => URI.encode_www_form(options))
	end

which is always using URI::HTTP.build to construct the URI object. I'm far from a Rubyist, yet looking at the docs here: https://ruby-doc.org/stdlib-2.6.3/libdoc/uri/rdoc/URI/HTTP.html :scheme does not matter as it expects you to use URI:HTTPS for constructing HTTPS URIs. This is also what I can see when testing this in irb:

➜  ~ irb
2.5.1 :001 > require 'uri'
 => false 
2.5.1 :002 > u = URI::HTTP.build(:scheme => 'https', :host => 'couchdb.example.com')
 => #<URI::HTTP http://couchdb.example.com> 
2.5.1 :003 > u.to_s
 => "http://couchdb.example.com" 
2.5.1 :004 > u = URI::HTTPS.build(:scheme => 'https', :host => 'couchdb.example.com')
 => #<URI::HTTPS https://couchdb.example.com> 
2.5.1 :005 > u.to_s
 => "https://couchdb.example.com" 

Is this a bug in the library and it should use either URI:HTTP or URI:HTTPS instead of passing :scheme or are we missing something else here?

[m90]

To make things even more complicated it looks like the behavior of URI::HTTP changed from Ruby 1 to Ruby 2 (where support for :scheme was dropped in Ruby 2).

We are using the docker.elastic.co/logstash/logstash:6.4.3 Docker image which runs:

bash-4.2$ ruby -v
jruby 9.1.13.0 (2.3.3) 2017-09-06 8e1c115 OpenJDK 64-Bit Server VM 25.191-b12 on 1.8.0_191-b12 +jit [linux-x86_64]

Running the above code sample in a container based off that image I can see the Ruby 2 behavior though:

➜  ~ docker run --rm -it docker.elastic.co/logstash/logstash:6.4.3 bash
bash-4.2$ ruby <<EOF
> require 'uri'
> u = URI::HTTP.build(:scheme => 'https', :host => 'couchdb.example.com')
> puts u.to_s
> u = URI::HTTPS.build(:scheme => 'https', :host => 'couchdb.example.com')
> puts u.to_s
> EOF
http://couchdb.example.com
https://couchdb.example.com
bash-4.2$ 

Is this plugin limited to a certain version range of Logstash and we'd need to downgrade?

[njuguna-n]

The last release was before a fix for this issue was merged. Is it possible to have a new release so that this can be closed?