Cryptography Error when running talker_listener on two machines

[victomteng1997]

Bug report

Required Info:

• Operating System:
  • Ubuntu 20.04
• Installation type:
  • Debian packages
• Version or commit hash:
  • followed the procedures in SROS2_Linux.md
• DDS implementation:
  • should be the default Fast-RTPS

Steps to reproduce issue

I followed the documentation listed in SROS2_Linux.md and everything works on single machine. Also tried to set up talker and listener on the two test machines without security features and the communication is good.

I then enabled security, and copied the keystore from one machine to another. I tried to copy files under talker folder only, and also tried to copy the whole keystore by scp. When I executed the talker on the remote machine:

$ ros2 run demo_nodes_py talker --ros-args --enclave /talker_listener/talker

[INFO] [1620197319.436928826] [rcl]: Found security directory: /home/user/sros2_demo/demo_keystore/enclaves/talker_listener/talker
2021-05-04 23:48:39.559 [SECURITY Error] Received Writer Cryptography message but not found local reader da.f7.10.ce.d9.77.51.2c.40.a4.95.2a|ff.1.1.c7 -> Function process_participant_volatile_message_secure
2021-05-04 23:48:39.559 [SECURITY Error] Received Reader Cryptography message but not found local writer da.f7.10.ce.d9.77.51.2c.40.a4.95.2a|ff.1.1.c2 -> Function process_participant_volatile_message_secure
[INFO] [1620197320.500136204] [talker]: Publishing: "Hello World: 0"
[INFO] [1620197321.492312859] [talker]: Publishing: "Hello World: 1"

On the listener side, there's no terminal output:

$ ros2 run demo_nodes_py listener --ros-args --enclave /talker_listener/listener

[INFO] [1620197312.966061319] [rcl]: Found security directory: /home/user/sros2_demo/demo_keystore/enclaves/talker_listener/listener

I also notice that the Security Error is raised when the listener is started. Tried to search for this error online but didn't find anything useful. Anyone can help to resolve?

[ruffsl]

When distributing enclaves within a keystore, you'll have to copy over the public folder in the keystore to the external hosts as well. Currently the enclave generation makes use of relative symlinks to point to the public certificates for the trusted certificate authorities. For your multi-machine use case, I assume you've verified that everything is working properly when security is at least disabled, ie your physical networking is set up correctly.

[victomteng1997]

I have verified that everything is working properly when security is disabled.
I noticed that the keystore use symlink to point to public certificates, but when using scp command, the target of symlink will be copied. I then tried to copy the whole keystore directory, including public folder to the remote machine, but the error message is the same.

Meanwhile, I also tried create an empty test_demo_keystore directory on the same machine where demo_keystore is created. I then copied the talker folder into this test_demo_keystore and everything worked. Don't know why it doesn't work on two machines though.

[ruffsl]

I have verified that everything is working properly when security is disabled.

Ok, that's a good start.

• Version or commit hash:

  • followed the procedures in SROS2_Linux.md

What version is this exactly? This isn't specific, so I'm not even sure what ROS2 distro or debian package version this is.

I then tried to copy the whole keystore directory, including public folder to the remote machine, but the error message is the same.

I'm not sure the issue here is with the keystore, but you could tarball the keystore before copying it if scp cant handle symlinks.

Don't know why it doesn't work on two machines though.

The security error log message seems to originate from here:

https://github.com/eProsima/Fast-DDS/blob/af466cfe63a8319cc9d37514267de8952627a9a4/src/cpp/rtps/security/SecurityManager.cpp#L1799

Perhaps you could verify that DDS security for your version of FastRTPS is working outside of ROS?

https://github.com/eProsima/Fast-DDS/tree/master/examples/C%2B%2B/SecureHelloWorldExample

If this SecureHelloWorldExample isn't working, then you may want to follow up with eProsima or try an alternate RMW.