Sometimes ROS_SECURITY_ENCLAVE_OVERRIDE does not effectively work as introduced

[fujitatomoya]

Bug report

Required Info:

• Operating System:
  • Ubuntu 24.04
• Installation type:
  • source build
• Version or commit hash:
  • ros2/ros2@4d07f58
• DDS implementation:
  • Fast-DDS likely, Cyclonedds sometimes
• Client library (if applicable):
  • rclpy, ros2cli

Steps to reproduce issue

https://github.com/ros2/sros2/blob/rolling/SROS2_Linux.md#run-the-demo-1 introduces ROS_SECURITY_ENCLAVE_OVERRIDE to be used with ros2cli such as ros2 node list and ros2 topic list.
But sometimes it does not print the available nodes or topics as explained.

• start talker and listener, make sure they are working as expected.

ros2 run demo_nodes_cpp talker --ros-args --enclave /talker_listener/talker
ros2 run demo_nodes_py listener --ros-args --enclave /talker_listener/listener

• call ros2cli as followings.

root@tomoyafujita:~/ros2_ws/colcon_ws# ros2 node list
root@tomoyafujita:~/ros2_ws/colcon_ws# ros2 topic list
/parameter_events
/rosout

• even with --no-daemon, it sometimes does not work.

root@51cdd59e1f3e:~# ros2 node list --no-daemon
[INFO] [1715030178.386225737] [rcl]: Found security directory: /root/sros2_demo/demo_keystore/enclaves/talker_listener/listener
root@51cdd59e1f3e:~# ros2 topic list --no-daemon
[INFO] [1715030183.341342733] [rcl]: Found security directory: /root/sros2_demo/demo_keystore/enclaves/talker_listener/listener
/parameter_events
/rosout

Expected behavior

node list and topic list should print available nodes (talker and listener) and topic (/chatter).

Actual behavior

described above.

Additional information

originally found on #295

[fujitatomoya]

So this is all about the ros2 daemon.

root@tomoyafujita:~/ros2_ws/colcon_ws# env | grep ROS
ROS_VERSION=2
ROS_SECURITY_ENCLAVE_OVERRIDE=/talker_listener/listener
ROS_PYTHON_VERSION=3
ROS_SECURITY_KEYSTORE=/root/sros2_demo/demo_keystore
ROS_AUTOMATIC_DISCOVERY_RANGE=SUBNET
ROS_SECURITY_STRATEGY=Enforce
ROS_DISTRO=rolling
ROS_SECURITY_ENABLE=true
root@tomoyafujita:~/ros2_ws/colcon_ws# ros2 daemon stop
The daemon is not running
root@tomoyafujita:~/ros2_ws/colcon_ws# ros2 daemon start
The daemon has been started
root@tomoyafujita:~/ros2_ws/colcon_ws# ros2 node list
/listener
/talker
root@tomoyafujita:~/ros2_ws/colcon_ws# ros2 topic list
/chatter
/parameter_events
/rosout

as you can see above, if ros2 daemon is already running, it does not have security enclaves, so that it cannot discover those secured apps in the network. restarting daemon can solve this problem. (this actually brings me a question, daemon should inherit the security enclaves silently? so that everyone can see the ROS 2 node and endpoint graph via xmlrpc. i think this is not supposed to be seen by unsecured or unauthorized users who do not have security enclaves?)

the other thing is, we should probably introduce the --no-daemon option in sros2 doc to avoid the above problem in 1st place.

root@tomoyafujita:~/ros2_ws/colcon_ws# ros2 node list --no-daemon --spin-time 5
[INFO] [1715191452.038336038] [rcl]: Found security directory: /root/sros2_demo/demo_keystore/enclaves/talker_listener/listener
/listener
/talker
root@tomoyafujita:~/ros2_ws/colcon_ws# ros2 topic list --no-daemon --spin-time 5
[INFO] [1715191462.846370212] [rcl]: Found security directory: /root/sros2_demo/demo_keystore/enclaves/talker_listener/listener
/chatter
/parameter_events
/rosout

--spin-time should be also explained to get the result with --no-daemon option.
if --spin-time is not specified, NodeStrategy uses 0.5 sec default for the discovery without daemon, sometimes it does not give it enough time when the security is enabled. (authorization, handshake and so on take time longer than 0.5sec, eventually NodeStrategy ends up having no graph information.)

[fujitatomoya]

CC: @mikaelarguedas just FYI.

[mikaelarguedas]

Thanks @fujitatomoya for testing more thoroughly and reporting!

There seem to be 3 different things:

• The overriding of enclaves seems to work as the ros2 tools (or other nodes with enclave overriden) always seem to look for the security files at the place dictated by the environment variable

• The leaking of information is another topic that comes down to multiple things:

  • DDS Spec issues: some pointed out by Ruffin here SROS2 leaks node information, regardless of rtps_protection_kind setup  #172 and related tickets
  • SROS2 default configuration (no easily modifiable): overall RTPS traffic is SIGNED and not encrypted

• The daemon behavior:

  • likely related to the point above. But may need more investigation as well as it seems that also without security the daemon surprises user expectation
  • 👍 for adjusting the docs with --no-daemon and --spin-time, this is actually how the sros2 test run to be less flaky due to this inconsistent daemon behavior:

    sros2/sros2/test/sros2/commands/security/verbs/utilities/sros2_cli_test_case.py

    Lines 72 to 74 in 07608ab

    	if not use_daemon:
    	# Wait for direct node to discover fixture nodes.
    	cmd.extend(['--no-daemon', '--spin-time', f'{MAX_DISCOVERY_DELAY}'])

[fujitatomoya]

@mikaelarguedas thanks for the information.

something i am concerned here is, daemon could be spawned with security enclaves and expose the discovery information for everyone. and it does that silently (authorized developer could easily expose secured information without knowing it once they use ros2cli commands) ... that is the major concern for 3rd party developer and end users for me. probably ros2 daemon should not inherit the security enclaves by default?

i guess this topic needs to be discussed on https://github.com/ros2/ros2cli though.

[mikaelarguedas]

daemon could be spawned with security enclaves and expose the discovery information for everyone. and it does that silently (authorized developer could easily expose secured information without knowing it once they use ros2cli commands)

I guess we'd need to dig a bit deeper with people knowing more about the daemon (either over at ros2cli or here) to figure out what is actually inherited from the parent process and figure out a way to mitigate it

[fujitatomoya]

CC: @mikaelarguedas

i want to keep the one issue for the fix, so i will go ahead to close this one. and create another issue to keep the discussion on ros2 daemon could expose secured network.