Cannot connect to SSL enabled Zookeeper

[jheinitz]

Hello,

I configured my Zookeeper and Kafka Broker to communicate over SSL using certificates signed by our company's CA. The problem is that I get an error while the zookeeper is queried for its metrics. This is the error message:

kafka-ui | 14:59:50.054 [kafka-admin-client-thread | adminclient-1] ERROR com.provectus.kafka.ui.service.ZookeeperService - A zookeeper exception has occurred
kafka-ui | org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /brokers/ids
kafka-ui | at org.apache.zookeeper.KeeperException.create(KeeperException.java:102) ~[zookeeper-3.5.7.jar!/:3.5.7]
kafka-ui | at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.5.7.jar!/:3.5.7]
kafka-ui | at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:2589) ~[zookeeper-3.5.7.jar!/:3.5.7]
kafka-ui | at com.provectus.kafka.ui.service.ZookeeperService.isZkClientConnected(ZookeeperService.java:37) ~[classes!/:?]
kafka-ui | at com.provectus.kafka.ui.service.ZookeeperService.isZookeeperOnline(ZookeeperService.java:29) ~[classes!/:?]
kafka-ui | at com.provectus.kafka.ui.service.KafkaService.buildFromData(KafkaService.java:174) ~[classes!/:?]
kafka-ui | at com.provectus.kafka.ui.service.KafkaService.lambda$getUpdatedCluster$3(KafkaService.java:142) ~[classes!/:?]

I get a similar error when I invoke the zookeeper-shell without specifying the zk-tls-config file. I also get an error in my zookeeper log indicating that a request was made which was not a SSL/TLS request. Log from Zookeeper (zookeeper-server.log):

[2021-11-01 13:09:09,079] ERROR Unsuccessful handshake with session 0x0 (org.apache.zookeeper.server.NettyServerCnxnFactory)
[2021-11-01 13:09:09,079] WARN Exception caught (org.apache.zookeeper.server.NettyServerCnxnFactory)
io.netty.handler.codec.DecoderException: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 0000002d0000000000000000000000000000ea600000000000000000000000100000000000000000000000000000000000

This makes me think that the Kafka-UI is not using SSL when connecting the zookeeper on :2182.

docker-compose .yml with configuration: docker-compose.yml.txt

I'm using my own docker image based on the official one. I'm copying the root CA certificate to the image and run update-ca-certificates and import it into the Java keystore using keytool. Please see my Dockerfile for details:
Dockerfile.txt

Maybe I missed something, but I fail to figure it out. If you have any further questions, please come back to me.

Thanks for your help in advance.

Best regards

Jens

[Haarolean]

Hi Jens, thanks for reaching out.
Actually we have a ready docker-compose example with SSL-enabled zookeeper, located here. Please try it out :)