How to get ua_class of DataType of specific NameSpace

[jcbastosportela]

Hi all. First of all, great work keeping this up to date with that standard!

I am using asyncua (0.9.0) for a tester client for a PLC that implements an OPC UA server. I am developing my own inf model.

The PLC creates its own OPC UA DataTypes on its own Namespace. I have my own inf model also loaded into that PLC, so it also has my defined DataTypes.

I have a method that takes one argument of a custom structured DataType, lets say LinearOutputDeviceConfigDataType.
In order to handle the method on the PLC I have to define a plc data type binary matching the OPC UA definition. The name that I give to the plc data type becomes a OPC UA DataType on the PLC's NameSpace. In my case I was giving the same name as on the Information Model I created. When from asyncua client I call the method giving as argument an object of the type get with get_ua_class('LinearOutputDeviceConfigDataType') apparently I am getting an instance of the DataType from the PLC's NameSpace, and it causes the PLC to reject the call.

When I rename the PLC's data type to LinearOutputDeviceConfigDT this works.

Consider the image bellow:

So, my question really is: is there a way of getting the ua_class of the specific Namespace? Or is there other way of getting the class of the input argument that makes sure that it is of the correct namespace?

Thank you so much for your help.

Regards,
Portela

[oroulet]

ahah I knew that one would pop up one day. you cannot...

[oroulet]

but I just made a PR: #508

[jcbastosportela]

but I just made a PR: #508

Very good man! Very fast.
I will try it as soon as possible.

[jcbastosportela]

I was using 0.9.0, and when I checkout this commit many new things are coming I believe. With this version I get an exception on load_data_type_definitions(). Opened a new issue for this #509 (comment)

[jcbastosportela]

So, yesterday after finding the issue I have tried some dirty hacks to get to try this functionality. Basically I have made similar changes (commit 4fb0d26), catching same exceptions, but passing instead of raising, only to get through that failing DataType, which in my case seems to be the only one causing issues.
I also changed common/structures104.py lines 381:383 to look like this:

        env = await _generate_object(name, edef, enum=True)
        try:
            ua.register_enum(name, desc.NodeId, env[name])
            new_enums[name] = env[name]
        except Exception:
            logger.exception(f'The {name} failled')
            pass

Just to be able to try your functionality to load a specific data type, but it didn't work for me. I get this:

Traceback (most recent call last):
  File "c:/BnR/exe_ftplc/Tests/regressionTests/client-data-type-definition.py", line 34, in <module>
    asyncio.run(main())
  File "C:\Program Files (x86)\Python37-32\lib\asyncio\runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "C:\Program Files (x86)\Python37-32\lib\asyncio\base_events.py", line 584, in run_until_complete
    return future.result()
  File "c:/BnR/exe_ftplc/Tests/regressionTests/client-data-type-definition.py", line 30, in main
    my_type = await load_custom_struct( mystructnode )
  File "c:\BnR\exe_ftplc\Tests\regressionTests\asyncua\common\structures104.py", line 293, in load_custom_struct
    sdef = await node.read_data_type_definition()
  File "c:\BnR\exe_ftplc\Tests\regressionTests\asyncua\common\node.py", line 153, in read_data_type_definition
    result = await self.read_attribute(ua.AttributeIds.DataTypeDefinition)
  File "c:\BnR\exe_ftplc\Tests\regressionTests\asyncua\common\node.py", line 301, in read_attribute
    result[0].StatusCode.check()
  File "c:\BnR\exe_ftplc\Tests\regressionTests\asyncua\ua\uatypes.py", line 335, in check
    raise UaStatusCodeError(self.value)
asyncua.ua.uaerrors._auto.BadAttributeIdInvalid: "The attribute is not supported for the specified Node."(BadAttributeIdInvalid)

But now I got a bit confused on how this is really supposed to work. The datatype I want to get is in fact a derived datatype from "My inf Model"; On the PLC I had to implement it flat (PLC datatypes can't inherit). Consider the image:

In this case is the following code correct? This is the code the outputs the error above.

async def main():
    url = 'opc.tcp://localhost:4848'
    async with Client(url=url) as client:
        uri = 'urn:asml:scanner:machine_conditioning:ftplc'
        idx = await client.register_namespace(uri)
        await client.load_data_type_definitions()

        # loading one specific custom struct
        mystructnode = await client.nodes.base_structure_type.get_child(f"{idx}:LinearInputDeviceConfigDataType") 
        # mystructnode = client.get_node( ua.NodeId(3005,idx) )   # results in same error
        my_type = await load_custom_struct( mystructnode )
        print(my_type)

Thank you.
Portela

[oroulet]

you may get that issue if your server does not support spec 1.4. Can you check if your datatypes node have an attribute callled DataTypeDefinition?

[jcbastosportela]

This is all I see when selecting the datatype:

NOTE: I can normally get python objects out of OPC UA datatypes from the server. both with get_ua_class(TYPE_NAME)() and ua.TYPE_NAME(). In my case when using this I am only getting the object of the wrong inf model (because there are 2 models which define a data type of the same name).

[oroulet]

OK. If your server support spec 1.4 Then you can use the load_data-type_definition() (return nothing) and load_custom_struct() (rturn the struct).
IF your server only support the old custom struct format then use the load_type_definition() it return many things, one of them is a dict of all structures, so you will find your structures there. You can also pass a specific node to load_type_definition() (one of the nodes under the "OPC Binary" node). It will work.

[jcbastosportela]

OK. If your server support spec 1.4 Then you can use the load_data-type_definition() (return nothing) and load_custom_struct() (rturn the struct).
IF your server only support the old custom struct format then use the load_type_definition() it return many things, one of them is a dict of all structures, so you will find your structures there. You can also pass a specific node to load_type_definition() (one of the nodes under the "OPC Binary" node). It will work.

Running with load_type_definitions

I am not sure if I understood this correctly. I don't see load_type_definition() method anymore. It was deprecated and aparently removed(?)
So, the method name is load_type_definitions with s XD. When I run with this I get some errors (works fine with 0.9.0, needs the buider <> builder fix :) ):

WARNING:asyncua.client.client:Deprecated since spec 1.04, call load_data_type_definitions
INFO:asyncua.client.ua_client.UaClient:browse


@dataclass
class FetchResultDataType:

    '''
    FetchResultDataType structure autogenerated from xml
    '''




@dataclass
class FetchResultErrorDataType:

    '''
    FetchResultErrorDataType structure autogenerated from xml
    '''

    Status:ua.Int32 = 0
    Diagnostics:ua.DiagnosticInfo = ua.DiagnosticInfo()



@dataclass
class FetchResultDataDataType:

    '''
    FetchResultDataDataType structure autogenerated from xml
    '''

    SequenceNumber:ua.Int32 = 0
    EndOfResults:ua.Boolean = True
    ParameterDefs:List[ua.ParameterResultDataType] = field(default_factory=list)

ERROR:asyncua.common.structures:Failed to execute auto-generated code from UA datatype: 

@dataclass
class FetchResultDataDataType:

    '''
    FetchResultDataDataType structure autogenerated from xml
    '''

    SequenceNumber:ua.Int32 = 0
    EndOfResults:ua.Boolean = True
    ParameterDefs:List[ua.ParameterResultDataType] = field(default_factory=list)
Traceback (most recent call last):
  File "c:\BnR\exe_ftplc\Tests\regressionTests\asyncua\common\structures.py", line 281, in _generate_python_class
    exec(code, env)
  File "<string>", line 3, in <module>
  File "<string>", line 12, in FetchResultDataDataType
AttributeError: module 'asyncua.ua' has no attribute 'ParameterResultDataType'
INFO:asyncua.client.client:disconnect
INFO:asyncua.client.ua_client.UaClient:close_session
INFO:asyncua.client.ua_client.UASocketProtocol:close_secure_channel
INFO:asyncua.client.ua_client.UASocketProtocol:Request to close socket received
INFO:asyncua.client.ua_client.UASocketProtocol:Socket has closed connection
Traceback (most recent call last):
  File "c:/BnR/exe_ftplc/Tests/regressionTests/client-data-type-definition.py", line 32, in <module>
    asyncio.run(main())
  File "C:\Program Files (x86)\Python37-32\lib\asyncio\runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "C:\Program Files (x86)\Python37-32\lib\asyncio\base_events.py", line 584, in run_until_complete
    return future.result()
  File "c:/BnR/exe_ftplc/Tests/regressionTests/client-data-type-definition.py", line 19, in main
    await client.load_type_definitions()
  File "c:\BnR\exe_ftplc\Tests\regressionTests\asyncua\client\client.py", line 618, in load_type_definitions
    return await load_type_definitions(self, nodes)
  File "c:\BnR\exe_ftplc\Tests\regressionTests\asyncua\common\structures.py", line 223, in load_type_definitions
    generator.get_python_classes(structs_dict)
  File "c:\BnR\exe_ftplc\Tests\regressionTests\asyncua\common\structures.py", line 177, in get_python_classes
    return _generate_python_class(self.model, env=env)
  File "c:\BnR\exe_ftplc\Tests\regressionTests\asyncua\common\structures.py", line 281, in _generate_python_class
    exec(code, env)
  File "<string>", line 3, in <module>
  File "<string>", line 12, in FetchResultDataDataType
AttributeError: module 'asyncua.ua' has no attribute 'ParameterResultDataType'

Running with load_data_type_definitions

Moreover, I see something "strange". When I run the load_data_type_definitions() it prints info log, but on that output I don't see all the DataTypes that are sub-types of QualifiedName(NamespaceIndex=9, Name='DeviceConfigBaseDataType') that are defined with the same name on other name space.

But for other DataType that same does not happen:
QualifiedName(NamespaceIndex=10, Name='MethodExecutionResultDataType')
and
QualifiedName(NamespaceIndex=6, Name='MethodExecutionResultDataType')
are both registered.

But using a OPC UA client I do see them on the browser:

log bellow:

INFO:asyncua.client.ua_client.UaClient:browse
INFO:asyncua.common.structures104:Registring data type NodeId(Identifier=6522, NamespaceIndex=2, NodeIdType=<NodeIdType.FourByte: 1>) QualifiedName(NamespaceIndex=2, Name='FetchResultDataType')
INFO:asyncua.common.structures104:Registring data type NodeId(Identifier=6525, NamespaceIndex=2, NodeIdType=<NodeIdType.FourByte: 1>) QualifiedName(NamespaceIndex=2, Name='ParameterResultDataType')
INFO:asyncua.common.structures104:Registring data type NodeId(Identifier=50010, NamespaceIndex=4, NodeIdType=<NodeIdType.FourByte: 1>) QualifiedName(NamespaceIndex=4, Name='BrAccessRightsDataType')
INFO:asyncua.common.structures104:Registring data type NodeId(Identifier=50040, NamespaceIndex=4, NodeIdType=<NodeIdType.FourByte: 1>) QualifiedName(NamespaceIndex=4, Name='BrAuthorizationDataType')
INFO:asyncua.common.structures104:Registring data type NodeId(Identifier=50030, NamespaceIndex=4, NodeIdType=<NodeIdType.FourByte: 1>) QualifiedName(NamespaceIndex=4, Name='BrNodeAccessDataType')
INFO:asyncua.common.structures104:Registring data type NodeId(Identifier=50020, NamespaceIndex=4, NodeIdType=<NodeIdType.FourByte: 1>) QualifiedName(NamespaceIndex=4, Name='BrRoleAccessDataType')
INFO:asyncua.common.structures104:Registring data type NodeId(Identifier=3002, NamespaceIndex=6, NodeIdType=<NodeIdType.FourByte: 1>) QualifiedName(NamespaceIndex=6, Name='MethodExecutionResultDataType')
INFO:asyncua.common.structures104:Registring data type NodeId(Identifier=3003, NamespaceIndex=6, NodeIdType=<NodeIdType.FourByte: 1>) QualifiedName(NamespaceIndex=6, Name='VersionDataType')
INFO:asyncua.common.structures104:Registring data type NodeId(Identifier=3003, NamespaceIndex=8, NodeIdType=<NodeIdType.FourByte: 1>) QualifiedName(NamespaceIndex=8, Name='SensorBaseDataType')
INFO:asyncua.common.structures104:Registring data type NodeId(Identifier=3002, NamespaceIndex=9, NodeIdType=<NodeIdType.FourByte: 1>) QualifiedName(NamespaceIndex=9, Name='DeviceConfigBaseDataType')
INFO:asyncua.common.structures104:Registring data type NodeId(Identifier=100000, NamespaceIndex=10, NodeIdType=<NodeIdType.Numeric: 2>) QualifiedName(NamespaceIndex=10, Name='LinearInputDeviceConfigDataType')
INFO:asyncua.common.structures104:Registring data type NodeId(Identifier=100010, NamespaceIndex=10, NodeIdType=<NodeIdType.Numeric: 2>) QualifiedName(NamespaceIndex=10, Name='MethodExecutionResultDataType')
INFO:asyncua.common.structures104:Registring data type NodeId(Identifier=100020, NamespaceIndex=10, NodeIdType=<NodeIdType.Numeric: 2>) QualifiedName(NamespaceIndex=10, Name='LinearOutputDeviceConfigDataType')
INFO:asyncua.common.structures104:Registring data type NodeId(Identifier=100030, NamespaceIndex=10, NodeIdType=<NodeIdType.Numeric: 2>) QualifiedName(NamespaceIndex=10, Name='NopInputDeviceConfigDataType')
INFO:asyncua.common.structures104:Registring data type NodeId(Identifier=100040, NamespaceIndex=10, NodeIdType=<NodeIdType.Numeric: 2>) QualifiedName(NamespaceIndex=10, Name='NopOutputDeviceConfigDataType')

EDIT:
So, after some debug, it seems to me that the problem is because in common/structures.py '_generate_python_class' the element.get_code() call returns code where one of the types depends on a type on the same code, but defined afterwards.

In this case, ParameterResultDataType is also on the code but after FetchResultDataDataType. Is it possible to split the code in chuncks and process one by one? Like this, a dirty hack could be to make N runs while the number of remaining codes keep decrementing, or equal to zero (if between loops the number of remaining codes remains the same means that there would be some issue, like unresolved dependency). After dinner I may try this myself :)

EDIT2:
So, I have made some very dirty hack in order to get this working (the load_type_definitions). I am too foreign to the inner details of this project in order to implement a correct fix. Basically what I did is, on common/structures.py:

• _generate_python_class: doesn't raise when a exec(code, env) fails, and adds the failed code to a list; this list is returned together with the env.
• load_type_definitions I have made a loop calling _generate_python_class for a model while there are failing codes (when the number o failed codes doesn't decrease between loops it raises exception)

This is far from ideal, but I couldn't see an easy way to fix this given the fact that the extensions on ua are registered with data from structs_dict after call to _generate_python_class. So, I guess that this "registration" must be done in _generate_python_class so the retry can be more localized. At least it can be made possible to only generate python classes for the missing data types, instead of the crazy brute force I made.

I have created a draft PR so you can see #517

It seems to me that the load_type_definitions will only keep one entry for a certain DataType name even if they are in different namespaces. Giving the NodeId of the DataTypes collection of the Namespace I want works, but simply calling load_type_definitions wont work, and I wasn't really aware of this. Well, this would work well enough for me, but now I am thinking, do I need to call load_type_definitions(NodeDataTypeCollectionNS) every time I want to access the python structure of the other namespace? Or can I simply keep the return from it for each of the nodes and keep using it?
Regards,
Portela