Uncaught InvalidSignature error

[ghost]

Hello,

I use python-opcua (0.98.3) to create a client which subscribes to datachanges on several nodes. (Thank you to every contributor for the work you put in this package by the way, it proves very useful !)

More often than not, after 7 or 8 minutes a problem occures where a signature fails to be validated and the library raise an InvalidSignature error.
I don't know if the signature error in itself comes from python-opcua or from the server (but it is not simply random so if someone else faces this problem you may consider creating a dedicated issue).

The main problem in my opinion is that this InvalidSignature exception is never caught by the library and since it is in a child thread, it's very hard to catch for the user. As a result, the child thread crashes, soon after that a second crash occures (TimeoutError) (not caught either), and the main thread is never notified that the connexion is no longer working.

Here are the two exceptions I receive:

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/user/.local/lib/python2.7/site-packages/opcua/client/ua_client.py", line 95, in _run
    self._receive()
  File "/home/user/.local/lib/python2.7/site-packages/opcua/client/ua_client.py", line 104, in _receive
    msg = self._connection.receive_from_socket(self._socket)
  File "/home/user/.local/lib/python2.7/site-packages/opcua/common/connection.py", line 309, in receive_from_socket
    return self.receive_from_header_and_body(header, ua.utils.Buffer(body))
  File "/home/user/.local/lib/python2.7/site-packages/opcua/common/connection.py", line 280, in receive_from_header_and_body
    header, body)
  File "/home/user/.local/lib/python2.7/site-packages/opcua/common/connection.py", line 55, in from_header_and_body
    crypto.verify(header_to_binary(obj.MessageHeader) + struct_to_binary(obj.SecurityHeader) + decrypted, signature)
  File "/home/user/.local/lib/python2.7/site-packages/opcua/crypto/security_policies.py", line 155, in verify
    self.Verifier.verify(data, sig)
  File "/home/user/.local/lib/python2.7/site-packages/opcua/crypto/security_policies.py", line 276, in verify
    raise uacrypto.InvalidSignature
InvalidSignature

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/home/user/.local/lib/python2.7/site-packages/opcua/client/client.py", line 62, in run
    val = server_state.get_value()
  File "/home/user/.local/lib/python2.7/site-packages/opcua/common/node.py", line 148, in get_value
    result = self.get_data_value()
  File "/home/user/.local/lib/python2.7/site-packages/opcua/common/node.py", line 157, in get_data_value
    return self.get_attribute(ua.AttributeIds.Value)
  File "/home/user/.local/lib/python2.7/site-packages/opcua/common/node.py", line 266, in get_attribute
    result = self.server.read(params)
  File "/home/user/.local/lib/python2.7/site-packages/opcua/client/ua_client.py", line 304, in read
    data = self._uasocket.send_request(request)
  File "/home/user/.local/lib/python2.7/site-packages/opcua/client/ua_client.py", line 77, in send_request
    data = future.result(self.timeout)
  File "/home/user/.local/lib/python2.7/site-packages/concurrent/futures/_base.py", line 464, in result
    raise TimeoutError()
TimeoutError

I managed to create a workaround where I catch exceptions from every thread in my software. But I think the library should definitely catch these exceptions (and really any exception raised in secondary threads) and offer some way for the user to detect that something wrong occured (maybe something as simple as a threading.Event).

[ghost]

Hi !

I did not actually solve the issue, however, I managed to create some kind of workaround.

I created a global excepthook (sys.excepthook) that catches concurrent.futures._base.TimeoutError and cryptography.exceptions.InvalidSignature.
Since threads are used you will find a second issue within python threading library this time so you need to use this : http://spyced.blogspot.com/2007/06/workaround-for-sysexcepthook-bug.html

I ignore InvalidSignature but for TimeoutError, I set a threading.Event (I cannot reset the connexion directly there since I am in the exception raised within the connexion)

Finally I have another thread that waits on the same threading.Event and whenever it is set, it resets the connexion.

I must admit it is quite ugly but it does the job...

[oroulet]

Can you try: #686

[ghost]

You need to replace except UaError: by except ua.UaError: in opcua/common/connection.py.

However, once it is done, it does not fix the issue (well it does but another one pops up)

Here is the output I get:

First I get an error message that seems legitimate

2018-08-29 13:14:08,642 [32173 - Thread-2] [connection.py:from_header_and_body:58] [ERROR] Could not verify signature for message MessageChunk(Header(type:MSG, chunk_type:F, body_size:148, channel:29773), SequenceHeader(SequenceNumber:None, RequestId:None ), SymmetricAlgorithmHeader(TokenId:11 ), 0 bytes)
Traceback (most recent call last):
  File "/lib_directory/opcua/common/connection.py", line 56, in from_header_and_body
    crypto.verify(header_to_binary(obj.MessageHeader) + struct_to_binary(obj.SecurityHeader) + decrypted, signature)
  File "/lib_directory/opcua/crypto/security_policies.py", line 155, in verify
    self.Verifier.verify(data, sig)
  File "/lib_directory/opcua/crypto/security_policies.py", line 276, in verify
    raise UaError("Invalid signature in data {} with signature {}".format(data, signature))
UaError: Invalid signature in data MSGF�Mt
                                          AB��S�X����),C����묄��@	��E���lY�?�8��%�8����t�b���аm�����Pո�5����<d���^���}%�Ǯ�{=iBM�e\��49��L�G�[�?E5�g�=5F�����
                                                                                                                                                                  bM� with signature ���
�ݏ
  ���7N�Q)��"�

then I get a bunch of

2018-08-29 13:14:08,728 [32173 - Thread-2] [ua_client.py:_run:100] [ERROR] Protocol Error
Traceback (most recent call last):
  File "/lib_directory/opcua/client/ua_client.py", line 95, in _run
    self._receive()
  File "/lib_directory/opcua/client/ua_client.py", line 104, in _receive
    msg = self._connection.receive_from_socket(self._socket)
  File "/lib_directory/opcua/common/connection.py", line 312, in receive_from_socket
    return self.receive_from_header_and_body(header, ua.utils.Buffer(body))
  File "/lib_directory/opcua/common/connection.py", line 283, in receive_from_header_and_body
    header, body)
  File "/lib_directory/opcua/common/connection.py", line 60, in from_header_and_body
    obj.SequenceHeader = struct_from_binary(ua.SequenceHeader, data)
  File "/lib_directory/opcua/ua/ua_binary.py", line 502, in struct_from_binary
    val = from_binary(uatype, data)
  File "/lib_directory/opcua/ua/ua_binary.py", line 479, in from_binary
    return unpack_uatype(vtype, data)
  File "/lib_directory/opcua/ua/ua_binary.py", line 197, in unpack_uatype
    return st.unpack(data)
  File "/lib_directory/opcua/ua/ua_binary.py", line 138, in unpack
    return struct.unpack(self.format, data.read(self.size))[0]
  File "/lib_directory/opcua/common/utils.py", line 65, in read
    raise NotEnoughData("Not enough data left in buffer, request for {0}, we have {1}".format(size, self))
NotEnoughData: Not enough data left in buffer, request for 4, we have Buffer(size:0, data:)

Then at some point TimeoutError is triggered and my workaround resets the connection

[PzthonNoob]

Hi, I have unfortunately the exact same Issue. I use opcua 0.98.6 and cryptography 2.5 and adjusted the things mentioned @ #686
The message I get after about 7 minutes is the one below. Does somebody know how to takle this issue?

Exception in thread Thread-2:
Traceback (most recent call last):
File "C:\Program Files\Python36\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "C:\Program Files\Python36\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "C:\Program Files\Python36\lib\site-packages\opcua\client\ua_client.py", line 96, in _run
self._receive()
File "C:\Program Files\Python36\lib\site-packages\opcua\client\ua_client.py", line 105, in _receive
msg = self._connection.receive_from_socket(self._socket)
File "C:\Program Files\Python36\lib\site-packages\opcua\common\connection.py", line 310, in receive_from_socket
return self.receive_from_header_and_body(header, ua.utils.Buffer(body))
File "C:\Program Files\Python36\lib\site-packages\opcua\common\connection.py", line 281, in receive_from_header_and_body
header, body)
File "C:\Program Files\Python36\lib\site-packages\opcua\common\connection.py", line 55, in from_header_and_body
crypto.verify(header_to_binary(obj.MessageHeader) + struct_to_binary(obj.SecurityHeader) + decrypted, signature)
File "C:\Program Files\Python36\lib\site-packages\opcua\crypto\security_policies.py", line 157, in verify
self.Verifier.verify(data, sig)
File "C:\Program Files\Python36\lib\site-packages\opcua\crypto\security_policies.py", line 278, in verify
raise uacrypto.InvalidSignature
cryptography.exceptions.InvalidSignature

[ebarrieau-pall]

I have the same issue and produce the same console results as @axelfaure when I make the changes proposed in #686 .

[Bnjmn83]

I am experiencing the very same issue after about 25 minutes being connected:

...
File "/lambda/opcua/crypto/security_policies.py", line 371, in verify. 
raise uacrypto.InvalidSignature. 
cryptography.exceptions.InvalidSignature.
...
File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 434, in result. 
raise TimeoutError(). 
concurrent.futures._base.TimeoutError.

What can I do about it?

I am runnning on version opcua-0.98.12