Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't link in /librcl_interfaces__rosidl_generator_py.so due to it not linking in libpython #119

Open
1 task done
tony-p opened this issue Dec 19, 2023 · 7 comments
Open
1 task done

Can't link in /librcl_interfaces__rosidl_generator_py.so due to it not linking in libpython #119

tony-p opened this issue Dec 19, 2023 · 7 comments
Labels
bug Something isn't working linux-64 needs-rebuild Issue has been fixed but packages require rebuild

Comments

@tony-p
Copy link

tony-p commented Dec 19, 2023

Solution to issue cannot be found in the documentation.

  • I checked the documentation.

Issue

I'm having an issue on linux (humble) that when it tries to link in the librcl_interfaces__rosidl_generator_py.so it is missing a number of symbols resulting in undefined reference symbols

env/lib/librcl_interfaces__rosidl_generator_py.so: undefined reference to `PyLong_AsSize_t'

It is essentially the same problem as this https://robotics.stackexchange.com/questions/25039/generated-rosidl-generator-py-so-files-issue

And it is the same for the robostack provided version, however the workarounds don't seem to work and I think it is because it is already too late and should have been applied to these packages or the ones in between.

$ ldd -r .pixi/env/lib/librcl_interfaces__rosidl_generator_py.so
	linux-vdso.so.1 (0x00007ffcd3fd2000)
	libbuiltin_interfaces__rosidl_generator_py.so => /home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so (0x00007f51715d8000)
	librcl_interfaces__rosidl_generator_c.so => /home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./librcl_interfaces__rosidl_generator_c.so (0x00007f51715be000)
	librosidl_runtime_c.so => /home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./librosidl_runtime_c.so (0x00007f51715b2000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f5171200000)
	libbuiltin_interfaces__rosidl_generator_c.so => /home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/././libbuiltin_interfaces__rosidl_generator_c.so (0x00007f517159b000)
	librcutils.so => /home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/././librcutils.so (0x00007f5171584000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f51715ef000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f517157f000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f517157a000)
undefined symbol: PyObject_GetAttrString	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_SetAttrString	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: _Py_Dealloc	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromLong	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsLong	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyImport_ImportModule	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CallObject	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsUnsignedLong	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromUnsignedLong	(/home/me/workspace/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyList_New	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromUnsignedLongLong	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBuffer_Release	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsUnsignedLongLong	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_GetAttrString	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyUnicode_AsUTF8String	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBytes_FromStringAndSize	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_SetAttrString	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBuffer_ToContiguous	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: _Py_Dealloc	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyExc_RuntimeError	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyErr_SetString	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyFloat_FromDouble	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PySequence_Size	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CheckBuffer	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBool_FromLong	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsSize_t	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyImport_ImportModule	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CallObject	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyList_SetItem	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CallFunctionObjArgs	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_Size	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyUnicode_DecodeUTF8	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: _Py_TrueStruct	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_GetBuffer	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsUnsignedLong	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromUnsignedLong	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsLongLong	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromLongLong	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PySequence_Fast	(.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)

Would patching the librcl_interfaces__rosidl_generator_py build to force link in libpython3 be the best solution here?

Installed packages

pixi Lock file is too long too paste here, will try find a more concise way to paste, but the relevant package is

- platform: linux-64
  name: ros-humble-rcl-interfaces
  version: 1.2.1
  category: main
  manager: conda
  dependencies:
  - libgcc-ng >=12
  - libgcc-ng >=12
  - libstdcxx-ng >=12
  - numpy >=1.21.6,<2.0a0
  - python
  - python_abi 3.10.* *_cp310
  - ros-humble-builtin-interfaces
  - ros-humble-ros-workspace
  - ros-humble-rosidl-default-runtime
  - ros2-distro-mutex 0.3.*
  - ros2-distro-mutex 0.3.* humble
  url: https://repo.prefix.dev/robostack-staging/linux-64/ros-humble-rcl-interfaces-1.2.1-py310h7c61026_3.tar.bz2
  hash:
    md5: 7123a8c0b2d39745f42ee72e444e8caf
    sha256: d66ad32b9a592a056b7b6a4eda2840a3b67d3b506a7ee3e29528bd81da9731c5
  build: py310h7c61026_3
  arch: x86_64
  subdir: linux-64
  build_number: 3
  size: 343519
  timestamp: 1675688249136

Environment info

pixi info
pixi 0.10.0

Platform            : linux-64
Virtual packages    : __unix=0=0
                    : __linux=6.2.0=0
                    : __glibc=2.35=0
                    : __archspec=1=x86_64
Cache dir           : /home/me/.cache/rattler/cache
Auth storage        : /home/me/.rattler/rattler_auth_store.json

Project
------------

Manifest file       : /home/me/workspace/project/pixi.toml
Dependency count    : 49
Last updated        : 19-12-2023 16:24:34
Target platforms    : win-64
                    : linux-64
@tony-p tony-p added the bug Something isn't working label Dec 19, 2023
@tony-p
Copy link
Author

tony-p commented Jan 3, 2024

Update: force linking in the output of $(python3-config --ldflags --embed) (from the linked issue) in the failing package does work, although as a temporary fix for another linking issue I had added the env/lib in LD_LIBRARY_PATH and then it tried to link in the wrong version of 'libfreebl3.so` which required a newer glibc

@Tobias-Fischer
Copy link
Contributor

Hi @tony-p - could you please try again with the recently built updated packages?

@tony-p
Copy link
Author

tony-p commented Feb 6, 2024

Tried again very naively and same issue occurs however checked and is taking build 3 instead of 5 of rcl-interfaces 1.2.1 so will look into a bit more when I have a little more time

@tony-p
Copy link
Author

tony-p commented Feb 7, 2024 •
edited
Loading

Dug a little deeper, and now have the build 5 and end up with the same errors. Further the ldd shows that these still aren't linked (below)

Further I encountered the following issues:

  • for some reason the dependency resolution was choosing an old version of sysroot that had libc v2.12. Forcing that sysroot >=2.17 solved it without other dependency conflicts so don't know why it wasn't chosen in the first place. This also means that there is the sysroot package and the virtual __glibc package both providing libc.
  • many ament_*lint executables are in the lib/<tool> directory which results in them not being found by cmake as they are not on the path. Manually adding each tool to the path (temporarily) solved that issue
ldd -r .pixi/env/lib/librcl_interfaces__rosidl_generator_py.so
        linux-vdso.so.1 (0x00007ffc8c0c7000)
        libbuiltin_interfaces__rosidl_generator_py.so => /home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so (0x00007f375c73b000)
        librcl_interfaces__rosidl_generator_c.so => /home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./librcl_interfaces__rosidl_generator_c.so (0x00007f375c721000)
        librosidl_runtime_c.so => /home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./librosidl_runtime_c.so (0x00007f375c715000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f375c4e6000)
        libbuiltin_interfaces__rosidl_generator_c.so => /home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/././libbuiltin_interfaces__rosidl_generator_c.so (0x00007f375c4de000)
        librcutils.so => /home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/././librcutils.so (0x00007f375c4c7000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f375c751000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f375c4c2000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f375c4bd000)
undefined symbol: PyObject_GetAttrString        (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_SetAttrString        (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: _Py_Dealloc   (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromLong       (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsLong (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyImport_ImportModule (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CallObject   (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsUnsignedLong (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromUnsignedLong       (/home/tonypaulussen/workspaces/Workcell-Automation-Core/.pixi/env/lib/./libbuiltin_interfaces__rosidl_generator_py.so)
undefined symbol: PyList_New    (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromUnsignedLongLong   (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBuffer_Release      (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsUnsignedLongLong     (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_GetAttrString        (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyUnicode_AsUTF8String        (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBytes_FromStringAndSize     (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_SetAttrString        (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBuffer_ToContiguous (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: _Py_Dealloc   (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyExc_RuntimeError    (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyErr_SetString       (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyFloat_FromDouble    (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PySequence_Size       (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CheckBuffer  (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyBool_FromLong       (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsSize_t       (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyImport_ImportModule (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CallObject   (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyList_SetItem        (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_CallFunctionObjArgs  (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_Size (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyUnicode_DecodeUTF8  (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: _Py_TrueStruct        (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyObject_GetBuffer    (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsUnsignedLong (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromUnsignedLong       (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_AsLongLong     (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PyLong_FromLongLong   (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)
undefined symbol: PySequence_Fast       (.pixi/env/lib/librcl_interfaces__rosidl_generator_py.so)

@Tobias-Fischer
Copy link
Contributor

I wonder if this should be public:

ros-humble/patch/ros-humble-rosidl-generator-py.patch

Line 61 in 52756fd

+target_link_libraries(${_target_name_lib} PRIVATE Python3::NumPy Python3::Module)

Any ideas @traversaro @wolfv?

@Tobias-Fischer
Copy link
Contributor

I've just pushed 7d0d8be which I have locally confirmed fixes this issue. If you rebuild ros-humble-rosidl-generator-py and then ros-humble-rcl-interfaces the issue will go away. Unfortunately at the moment I have not the capacity to build this in CI; but it would be easy to do (please see Contributing guidelines if you'd like to open a PR @tony-p).

@tony-p
Copy link
Author

tony-p commented Feb 28, 2024

Great thanks, will try find some time to try soon (also limited capacity) and if fixes and have time will definitely look into the CI build

@Tobias-Fischer Tobias-Fischer added the needs-rebuild Issue has been fixed but packages require rebuild label Feb 29, 2024
@tony-p tony-p mentioned this issue Apr 5, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working linux-64 needs-rebuild Issue has been fixed but packages require rebuild
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Tobias-Fischer @tony-p