[Bug][OPC-UA]: org.apache.plc4x.java.spi.generation.SerializationException: Error writing signed byte

[axelweinz]

What happened?

Specifying the server certificate file in the connecting string results in org.apache.plc4x.java.spi.generation.SerializationException: Error writing signed byte.

The server certificate file format is .der.

String connectionString = "opcua:tcp://localhost:4841?discovery=false&username=admin&password=admin&server-certificate-file=PATH_TO_SERVER_CERTIFICATE_FILE.der&security-policy=Basic128Rsa15"
try (PlcConnection plcConnection = PlcDriverManager.getDefault().getConnectionManager().getConnection(connectionString)) {
  ....
}

Here is the stack trace:
java.util.concurrent.CompletionException: java.lang.RuntimeException: org.apache.plc4x.java.spi.generation.SerializationException: Error writing signed byte\r\n\tat java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)\r\n\tat java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)\r\n\tat java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1159)\r\n\tat java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)\r\n\tat java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2179)\r\n\tat org.apache.plc4x.java.opcua.context.Conversation.lambda$4(Conversation.java:198)\r\n\tat java.base/java.util.function.Consumer.lambda$andThen$0(Consumer.java:65)\r\n\tat org.apache.plc4x.java.spi.Plc4xNettyWrapper.decode(Plc4xNettyWrapper.java:183)\r\n\tat io.netty.handler.codec.MessageToMessageCodec$2.decode(MessageToMessageCodec.java:81)\r\n\tat io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)\r\n\tat io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)\r\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)\r\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\r\n\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\r\n\tat io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)\r\n\tat io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)\r\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)\r\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\r\n\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\r\n\tat io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)\r\n\tat io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)\r\n\tat io.netty.handler.codec.ByteToMessageCodec.channelRead(ByteToMessageCodec.java:103)\r\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)\r\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\r\n\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\r\n\tat io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1407)\r\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)\r\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\r\n\tat io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:918)\r\n\tat io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)\r\n\tat io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)\r\n\tat io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)\r\n\tat io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)\r\n\tat io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)\r\n\tat io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)\r\n\tat io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\r\n\tat io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\r\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\r\nCaused by: java.lang.RuntimeException: org.apache.plc4x.java.spi.generation.SerializationException: Error writing signed byte\r\n\tat org.apache.plc4x.java.opcua.context.BaseEncryptionHandler.encodeMessage(BaseEncryptionHandler.java:121)\r\n\tat org.apache.plc4x.java.opcua.context.EncryptionHandler.encodeMessage(EncryptionHandler.java:73)\r\n\tat org.apache.plc4x.java.opcua.context.Conversation.request(Conversation.java:237)\r\n\tat org.apache.plc4x.java.opcua.context.Conversation.requestChannelOpen(Conversation.java:204)\r\n\tat org.apache.plc4x.java.opcua.context.SecureChannel.onConnectOpenSecureChannel(SecureChannel.java:209)\r\n\tat org.apache.plc4x.java.opcua.context.SecureChannel.lambda$1(SecureChannel.java:153)\r\n\tat java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1150)\r\n\t... 35 more\r\nCaused by: org.apache.plc4x.java.spi.generation.SerializationException: Error writing signed byte\r\n\tat org.apache.plc4x.java.spi.generation.WriteBufferByteBased.writeSignedByte(WriteBufferByteBased.java:342)\r\n\tat org.apache.plc4x.java.spi.generation.WriteBufferByteBased.writeByteArray(WriteBufferByteBased.java:95)\r\n\tat org.apache.plc4x.java.opcua.context.BaseEncryptionHandler.encodeMessage(BaseEncryptionHandler.java:108)\r\n\t... 41 more\r\nCaused by: java.nio.BufferOverflowException\r\n\tat java.base/java.nio.Buffer.nextPutIndex(Buffer.java:736)\r\n\tat java.base/java.nio.HeapByteBuffer.put(HeapByteBuffer.java:216)\r\n\tat com.github.jinahya.bit.io.BufferByteOutput.write(BufferByteOutput.java:127)\r\n\tat com.github.jinahya.bit.io.DefaultBitOutput.write(DefaultBitOutput.java:56)\r\n\tat com.github.jinahya.bit.io.AbstractBitOutput.unsigned8(AbstractBitOutput.java:72)\r\n\tat com.github.jinahya.bit.io.AbstractBitOutput.unsigned16(AbstractBitOutput.java:94)\r\n\tat com.github.jinahya.bit.io.AbstractBitOutput.writeInt(AbstractBitOutput.java:120)\r\n\tat com.github.jinahya.bit.io.AbstractBitOutput.writeByte(AbstractBitOutput.java:106)\r\n\tat org.apache.plc4x.java.spi.generation.WriteBufferByteBased.writeSignedByte(WriteBufferByteBased.java:340)\r\n\t... 43 more\r\n

If I omit the server-certificate-file parameter, I instead run into this error:
java.util.concurrent.CompletionException: java.lang.RuntimeException: java.lang.NullPointerException: Cannot invoke \"org.apache.plc4x.java.opcua.readwrite.PascalByteString.getLengthInBits()\" because \"this.receiverCertificateThumbprint\" is null

Version

v0.12.0

Programming Languages

• plc4j
• plc4go
• plc4c
• plc4net

Protocols

• AB-Ethernet
• ADS /AMS
• BACnet/IP
• CANopen
• DeltaV
• DF1
• EtherNet/IP
• Firmata
• KNXnet/IP
• Modbus
• OPC-UA
• S7

[splatch]

Hello @axelweinz, looks like for some reason generated message exceeds expected payload size. Are you testing v0.12?

[axelweinz]

Yes, I am using these two dependencies:

[splatch]

@axelweinz I've had a look on some manual tests I conducted. With prosys I have static (discovery=false) tests which specify both client and server certificate. I've made attempt to specify username/password and prosys refuses connection attempt, thus I can't reproduce yet your issue with buffer overflow.

When no client certificate is provided client will generate temporary self-signed cert. Are you sure your server accepts such certificate?

[axelweinz]

Thanks for looking into this. I have tried with both providing my own certificate and letting the client generate a temporary one. Both results in the same error. I have verified on the server side that in both cases, the client certificate does not show up on the server. Therefore it seems that the error happens when the client is trying to read the server certificate.

[splatch]

Could you please generate similar certificate for test and share it? I'd like to make a reproducer of this issue. Without it, it might be quite hard to spot cause and solve it. Since issue happens in encryption handler, very likely we calculate wrong size of payload and allocate too small buffer.
Earlier on I made tests with certificates having keys composed of 1024, 2048, 3072, 4096 and 5120 bytes. How long is key in server certificate you use?

[axelweinz]

Yes, here is the server certificate (0A8E5BC293F3319FF153ECA240E51E92323C3ABC) and the client certificate, key and truststore I have tested with. The client certificate and truststore are generated using OpenSSL, like this:

req -x509 -newkey rsa:2048 -keyout clientkey.pem -out clientcert.der -days 3652
pkcs12 -export -in clientcert.der -inkey clientkey.pem -out store.pfx

I set the password to "admin" for both the private key and the truststore.
The OPC server exposes both Basic128Rsa15 and Basic256. Both gives the same error.
certificates.zip

[fernandokendy]

I'm facing the same error:
Cannot invoke "org.apache.plc4x.java.opcua.readwrite.PascalByteString.getLengthInBits()" because "this.receiverCertificateThumbprint" is null
I first tested connecting with the certificate through OPCUA to Prosys OPCUA Simu server, which worked. When testing with plc4j, this error keep on popping. I created the keystore .p12 based on the working certificate.
When connecting as anonymous or with User and Password, it works fine.
try (PlcConnection plcConnection = driverManager.getConnection("opcua:tcp://Turing.lab.mtu-digilab.io:53530/OPCUA?" + "discovery=false&message-security=SIGN_ENCRYPT&security-policy=Basic256Sha256&" + "key-store-file=keystore.p12&key-store-password=admin")) ...
I'm a bit unexperienced with OPCUA and certificates.
I'll also upload my cert files.
cert.zip
Thank you :)

[splatch]

@axelweinz I had a look on your certs and for some reason our client logic calculates wrong payload size. I thought it might be configuration issue, but clearly our logic calculates buffer size too small to fit padding and signature data. Your server is innocent.
@fernandokendy Your seed files look fine, but if you disable discovery (discovery=false) then you must provide server-certificate-file parameter. That's why you have null for receiverCertificateThumbprint. It is not set by you nor library.

[axelweinz]

Thanks for confirming this @splatch. Do you know when this will be fixed or if there is a workaround?

[splatch]

@axelweinz I have to look closer to give you advice. For now I can just confirm issue is present and it is not environment specific.

[splatch]

One thing I've found while checking #1802 - currently our encryption logic require same length for client and server keys. If you face an issue with SIGN_ENCRYPT messaging mode, please make sure that client private key is same length as server. At least for now until I figure out where this issue comes from.

[axelweinz]

Interesting, this is likely the case for me since the server I'm connecting to is only exposing SIGN_ENCRYPT

[splatch]

@axelweinz Please try current develop (0.13-SNAPSHOT) to see if it improves situation.

[axelweinz]

Will do as soon as I have time!