Undefined reference to "caml_startup" when trying to run `dune utop`

[afdw]

In an ocaml-rs project, for example ocaml-rust-starter, running dune utop gives the following error (while linking):

/usr/bin/ld: src/libocaml_rust_starter.a(ocaml_interop-4a398e80f3ca6ceb.ocaml_interop.8eq22uyz-cgu.12.rcgu.o): in function `_ZN3std4sync4once4Once9call_once28_$u7b$$u7b$closure$u7d$$u7d$17h7171c1b9a6fc7898E.llvm.3140953836603515050':
ocaml_interop.8eq22uyz-cgu.12:(.text._ZN3std4sync4once4Once9call_once28_$u7b$$u7b$closure$u7d$$u7d$17h7171c1b9a6fc7898E.llvm.3140953836603515050+0x3b): undefined reference to `caml_startup'
/usr/bin/ld: src/libocaml_rust_starter.a(ocaml_interop-4a398e80f3ca6ceb.ocaml_interop.8eq22uyz-cgu.12.rcgu.o): in function `_ZN4core3ops8function6FnOnce40call_once$u7b$$u7b$vtable.shim$u7d$$u7d$17h59471d095cabe6c5E.llvm.3140953836603515050':
ocaml_interop.8eq22uyz-cgu.12:(.text._ZN4core3ops8function6FnOnce40call_once$u7b$$u7b$vtable.shim$u7d$$u7d$17h59471d095cabe6c5E.llvm.3140953836603515050+0x3b): undefined reference to `caml_startup'
collect2: error: ld returned 1 exit status

Adding this function somewhere where the linker sees it, for example using the following code, "fixes" the issue and utop starts fine:

#[ocaml::func]
pub fn caml_startup() {}

OCaml version: 4.11.1
Rust version: nightly
GCC version: 10.2.0

[zshipko]

Thanks for the report! I will be able to take a closer look in the next few days.

It looks like caml_startup is getting optimized out at link time - I know it is defined in ocaml_sys.

[zshipko]

Hm, It looks like this may be an issue with ocaml-interop - I will open an issue there and keep investigating.

[tizoc]

Any idea of what dune utop does to link stuff? ocaml-interop makes all calls to the OCaml runtime through ocaml-sys, and doesn't do anything related to linking because that is up to the consumer program (which may be a Rust or OCaml program or library).

[zshipko]

I will need to look into what dune utop is actually doing.

Do you use ocaml-interop with the bytecode interpreter? I wonder if this could be due to some initialization difference there?

If I run dune runtest in the ocaml-rust-starter repo with (modes byte) set on the test stanza I get the same linker error as above. I don't know much about the bytecode runtime but I would assume that caml_startup is defined there as well, but maybe that's not correct

[tizoc]

@zshipko yes, it is defined there too, bytecode also uses caml_startup. What I am thinking is that dune utop is probably producing a .so, but because it is meant to be loaded by the bytecode runtime, then the runtime is not included in the .so itself. This means that ocaml-interop references caml_startup but the final object into which it is being linked to doesn't include it.

I have to reflect more on it, but I think the correct thing would be for ocaml-interop to have a feature flag that disables the compilation of the runtime initialization code to cover cases like this one. What I am not sure about is what would be the proper way to have dune utop enable that feature flag.

[afdw]

I have to reflect more on it, but I think the correct thing would be for ocaml-interop to have a feature flag that disables the compilation of the runtime initialization code to cover cases like this one. What I am not sure about is what would be the proper way to have dune utop enable that feature flag.

Just a thought, I do know what is really going on: both static library and dynamic library are built; maybe the library type should determine the behavior here?

[tizoc]

Probably not, because I use ocaml-interop to produce a library that is dynamically linked into a Rust program, and that library also includes the OCaml runtime. But to be honest, the reason I produce a dynamic library and not a static one is that I haven't figure out how to make dune produce a static library for me to use.

[tizoc]

But my point is that when producing a .so you may want to include the OCaml runtime (along with other OCaml code) in it, so knowing that it is a .so is not enough.

[mimoo]

Anybody has found a work around for this?

[afdw]

Anybody has found a work around for this?

There is partial one is the description of the issue which work for me.

[mimoo]

This is what I get when I use your trick:

ld: warning: directory not found for option '-L/opt/local/lib'
duplicate symbol '_caml_startup' in:
    /xxx/_opam/lib/ocaml/libasmrun.a(startup_nat_n.o)
    src/lib/marlin_plonk_bindings/stubs/libmarlin_plonk_stubs.a(marlin_plonk_stubs-6a29015a694fc5ab.marlin_plonk_stubs.77bua11a-cgu.15.rcgu.o)
ld: 1 duplicate symbol for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
File "caml_startup", line 1:
Error: Error during linking (exit code 1)

[tizoc]

FYI @mrmr1993 made a PR that should solve this https://github.com/tezedge/ocaml-interop/pull/36

Once all is ready I will merge and make a new ocaml-interop release. @zshipko you will then need to bump the dependency version in ocaml-rs.

[zshipko]

@tizoc Cool, thanks for the update! I will make sure to follow that PR

[mrmr1993]

@zshipko I have a fix that I'm planning to roll out to MinaProtocol/mina when I get the time in the next few days (backed by the changes in the PR above). Is it useful to PR here yet, or would you prefer to wait / implement yourself once it's ready?