Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Linking error when building //oak_containers/syslogd:syslogd #5050

Open
hvolkmer opened this issue Mar 5, 2025 · 7 comments
Open

Linking error when building //oak_containers/syslogd:syslogd #5050

hvolkmer opened this issue Mar 5, 2025 · 7 comments

Comments

@hvolkmer
Copy link

hvolkmer commented Mar 5, 2025

Based on latest checkout from today:

$ nix develop .#githubBuildShell --command just oak_containers_tests

(or)

$ nix develop .#githubBuildShell --command bazel build //oak_containers/syslogd:syslogd

[...]
  = note: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: warning: libcap.so.2, needed by /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so, not found (try using -rpath or -rpath-link)
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: warning: libgcrypt.so.20, needed by /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so, not found (try using -rpath or -rpath-link)
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: warning: liblzma.so.5, needed by /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so, not found (try using -rpath or -rpath-link)
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: warning: libzstd.so.1, needed by /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so, not found (try using -rpath or -rpath-link)
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: warning: liblz4.so.1, needed by /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so, not found (try using -rpath or -rpath-link)
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `gcry_md_read@GCRYPT_1.6'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `lzma_stream_decoder@XZ_5.0'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `gcry_md_close@GCRYPT_1.6'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `gcry_md_setkey@GCRYPT_1.6'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `ZSTD_DStreamOutSize'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `gcry_md_copy@GCRYPT_1.6'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `gcry_md_ctl@GCRYPT_1.6'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `LZ4_versionNumber'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `gcry_md_open@GCRYPT_1.6'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `ZSTD_getFrameContentSize'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `gcry_control@GCRYPT_1.6'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `ZSTD_decompressStream'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `lzma_code@XZ_5.0'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `gcry_md_write@GCRYPT_1.6'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `ZSTD_freeDCtx'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `ZSTD_createDCtx'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `ZSTD_isError'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `LZ4_decompress_safe'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `gcry_md_get_algo_dlen@GCRYPT_1.6'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `gcry_md_reset@GCRYPT_1.6'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `LZ4_decompress_safe_partial'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `ZSTD_getErrorCode'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `lzma_end@XZ_5.0'
          /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: /home/hvolkmer/.cache/bazel/_bazel_hvolkmer/4c21be3a181200ae522617c823dd8903/external/oak_cc_toolchain_sysroot//usr/lib/x86_64-linux-gnu/libsystemd.so: undefined reference to `gcry_check_version@GCRYPT_1.6'
          collect2: error: ld returned 1 exit status
[...]

This can be fixed by applying:

diff --git i/oak_containers/syslogd/BUILD w/oak_containers/syslogd/BUILD
index 2940e2ce0d..d2acacae71 100644
--- i/oak_containers/syslogd/BUILD
+++ w/oak_containers/syslogd/BUILD
@@ -38,6 +38,12 @@ rust_binary(
         "@oak_crates_index//:signal-hook-tokio",
         "@oak_crates_index//:tokio",
         "@oak_crates_index//:tokio-stream",
+         ":zstd",
+         ":cap",
+         ":lzma",
+         ":lz4",
+         ":gcrypt",
+         ":gpg-error",
     ],
 )

@@ -45,3 +51,50 @@ alias(
     name = "syslogd",
     actual = ":oak_containers_syslogd",
 )
+
+cc_library(
+  name ="zstd",
+  srcs=[],
+  hdrs=[],
+  linkopts = ["-lzstd"],
+  visibility = ["//visibility:public"],
+)
+
+cc_library(
+  name ="cap",
+  srcs=[],
+  hdrs=[],
+  linkopts = ["-lcap"],
+  visibility = ["//visibility:public"],
+)
+
+cc_library(
+  name ="lzma",
+  srcs=[],
+  hdrs=[],
+  linkopts = ["-llzma"],
+  visibility = ["//visibility:public"],
+)
+
+cc_library(
+  name ="lz4",
+  srcs=[],
+  hdrs=[],
+  linkopts = ["-llz4"],
+  visibility = ["//visibility:public"],
+)
+cc_library(
+  name ="gcrypt",
+  srcs=[],
+  hdrs=[],
+   linkopts = ["-lgcrypt"],
+  visibility = ["//visibility:public"],
+)
+
+cc_library(
+  name ="gpg-error",
+  srcs=[],
+  hdrs=[],
+  linkopts = ["-lgpg-error"],
+  visibility = ["//visibility:public"],
+)

Is there a better way of fixing this?
@conradgrobler
Copy link
Collaborator

I am unable to reproduce this locally using the latest commit. The missing references should all be in the sysroot of the toolchain, so I am not sure what is causing this on your side.

What version of Bazel are you using?

Also, if you have Docker installed could you try running the following to see whether that works for you?

./scripts/docker_pull
./scripts/docker_run nix develop .#githubBuildShell --command bazel build //oak_containers/syslogd:syslogd

@hvolkmer
Copy link
Author

Bazel version:

$ nix develop .#githubBuildShell --command bazel version
warning: Git tree '/home/hvolkmer/oak' is dirty
warning: input 'crane' has an override for a non-existent input 'nixpkgs'
Starting local Bazel server and connecting to it...
Build label: 6.5.0- (@non-git)
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jan 1 00:00:00 1980 (315532800)
Build timestamp: 315532800
Build timestamp as int: 315532800

Running this via docker still gives the same result:

$ ./scripts/docker_run nix develop .#githubBuildShell --command bazel build //oak_containers/syslogd:syslogd
[...]
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
error: linking with `external/gcc_toolchain_x86_64/bin/gcc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="bazel-out/k8-fastbuild/bin/external/rust_linux_x86_64__x86_64-unknown-linux-gnu__nightly_tools/rust_toolchain/lib/rustlib/x86_64-unknown-linux-gnu/bin:bazel-out/k8-fastbuild/bin/ext
  = note: /root/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: cannot find -lzstd: No such file or directory
          /root/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: cannot find -lcap: No such file or directory
          /root/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: cannot find -llzma: No such file or directory
          /root/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: cannot find -llz4: No such file or directory
          /root/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: cannot find -lgcrypt: No such file or directory
          /root/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-ld: cannot find -lgpg-error: No such file or directory
          collect2: error: ld returned 1 exit status

[...]

The lib files are indeed there, it just seems like the linker isn't finding them.

@hvolkmer
Copy link
Author

I did a clean checkout and then the docker build worked, using "nix develop .#githubBuildShell --command bazel build //oak_containers/syslogd:syslogd" without docker still shows the error.

@conradgrobler
Copy link
Collaborator

Based on the warning above (warning: input 'crane' has an override for a non-existent input 'nixpkgs') I think the issue might be with your nix configuration/installation/version.

What version of nix are you using? What OS are you using on your development machine? Did you follow the nix installation instructions on https://github.com/project-oak/oak/blob/main/docs/development.md#install-nix?

@hvolkmer
Copy link
Author

I tried this on two machines:

  • my work box: Centos 9 + nix install as per the doc
  • my home box: Nixos 24.11 + nix install as per the doc

@conradgrobler
Copy link
Collaborator

That is suprising. My best guess is that something from the host OS is leaking into the build process because of nix develop. All the test machines I have available are running Debian. The Docker container we use and the toolchain sysroot are also based on Debian, which is presumably why it works consistently. I will try to find other ways to reproduce, but in the meantime I would recommend using the docker_run script for the builds to avoid the issue.

@hvolkmer
Copy link
Author

The docker build works as a workaround but it's very strange. The files exist, it's just the linker doesn't find them. See my initial comment. If you mess with the BUILD file it finds them (but I don't think that should be necessary). Bazel versions are the same and I don't see how OS environment via nix develop would influence the linker arguments/env - but that does seem to be what is happening.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@hvolkmer @conradgrobler