This is the bug Dan experienced and reported in Slack.

Dan used his own windows CA. He had no issues importing it, but afterwards the data node does not start successfully.

Expected Behavior

Current Behavior

The data node does not start successfully after the CA import, instead it throws errors:

Index cds_4 migration failed after 0 seconds: GetTaskResponse[completed=true, task=Task[node=jMDF5RCbRSKzHnzcv4i1eA, id=8234, type=transport, action=indices:data/write/reindex, status=TaskStatus[total=0, updated=0, created=0, deleted=0, batches=0, versionConflicts=0, noops=0, failures=null], description=reindex from [scheme=https host=glos01.eclipsenetwork.org port=9200 pathPrefix=/ query={ "match_all" : { "boost" : 1.0 } } username=elastic password=<<>>][cds_4] to [cds_4], startTimeInMillis=1719421000765, runningTimeInNanos=350832478, cancellable=true, cancelled=false, headers={X-Opaque-Id=667c47ce66e1a566a1b983a9}], error=type='s_s_l_handshake_exception', reason='PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target', causedBy='{type=validator_exception, reason=PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target, caused_by={type=sun_cert_path_builder_exception, reason=unable to find valid certification path to requested target}}'].

Seems like the certificate is the culprit, there's a certificate_unknown in its output:

2024-06-26T18:02:25.450Z INFO [OpensearchProcessImpl] [2024-06-26T18:02:25,448][ERROR][o.o.s.s.h.n.SecuritySSLNettyHttpServerTransport] [gldn03.lab.eclipsenetwork.org] Exception during establishing a SSL connection: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown 2024-06-26T18:02:25.450Z INFO [OpensearchProcessImpl] javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown 2024-06-26T18:02:25.450Z INFO [OpensearchProcessImpl] at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:130) ~[?:?] 2024-06-26T18:02:25.450Z INFO [OpensearchProcessImpl] at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117) ~[?:?] 2024-06-26T18:02:25.450Z INFO [OpensearchProcessImpl] at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:365) ~[?:?] 2024-06-26T18:02:25.450Z INFO [OpensearchProcessImpl] at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:287) ~[?:?] 2024-06-26T18:02:25.451Z INFO [OpensearchProcessImpl] at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:204) ~[?:?] 2024-06-26T18:02:25.451Z INFO [OpensearchProcessImpl] at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172) ~[?:?] 2024-06-26T18:02:25.451Z INFO [OpensearchProcessImpl] at java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:736) ~[?:?] 2024-06-26T18:02:25.451Z INFO [OpensearchProcessImpl] at java.base/sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:691) ~[?:?] 2024-06-26T18:02:25.451Z INFO [OpensearchProcessImpl] at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:506) ~[?:?] 2024-06-26T18:02:25.451Z INFO [OpensearchProcessImpl] at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:482) ~[?:?] 2024-06-26T18:02:25.451Z INFO [OpensearchProcessImpl] at java.base/javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:679) ~[?:?] 2024-06-26T18:02:25.451Z INFO [OpensearchProcessImpl] at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:310) ~[netty-handler-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.452Z INFO [OpensearchProcessImpl] at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1445) ~[netty-handler-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.452Z INFO [OpensearchProcessImpl] at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1338) ~[netty-handler-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.452Z INFO [OpensearchProcessImpl] at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1387) ~[netty-handler-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.452Z INFO [OpensearchProcessImpl] at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529) ~[netty-codec-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.452Z INFO [OpensearchProcessImpl] at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) ~[netty-codec-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.452Z INFO [OpensearchProcessImpl] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) ~[netty-codec-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.452Z INFO [OpensearchProcessImpl] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.452Z INFO [OpensearchProcessImpl] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.453Z INFO [OpensearchProcessImpl] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.453Z INFO [OpensearchProcessImpl] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.453Z INFO [OpensearchProcessImpl] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) [netty-transport-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.453Z INFO [OpensearchProcessImpl] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.453Z INFO [OpensearchProcessImpl] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.453Z INFO [OpensearchProcessImpl] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.453Z INFO [OpensearchProcessImpl] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) [netty-transport-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.453Z INFO [OpensearchProcessImpl] at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689) [netty-transport-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.453Z INFO [OpensearchProcessImpl] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652) [netty-transport-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.453Z INFO [OpensearchProcessImpl] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) [netty-transport-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.454Z INFO [OpensearchProcessImpl] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.454Z INFO [OpensearchProcessImpl] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.106.Final.jar:4.1.106.Final] 2024-06-26T18:02:25.454Z INFO [OpensearchProcessImpl] at java.base/java.lang.Thread.run(Thread.java:1583) [?:?] 2024-06-26T18:02:25.454Z INFO [OpensearchProcessImpl] [2024-06-26T18:02:25,450][WARN ][o.o.h.AbstractHttpServerTransport] [gldn03.lab.eclipsenetwork.org] caught exception while handling client http traffic, closing connection Netty4HttpChannel{localAddress=/192.168.99.92:9200, remoteAddress=/192.168.1.231:40352}

Possible Solution

Steps to Reproduce (for bugs)

1. Start migration (remote reindexing)
2. Upload own CA
3. Data node won't start up successfully.

Context

Migration testing.

Your Environment

• Graylog Version: 6.1 alpha 3
• Java Version:
• OpenSearch Version:
• MongoDB Version:
• Operating System:
• Browser version: