Prevent needless symbol exports

[FGasper]

From DCIT/perl-CryptX#79 (comment) …

Right now EU::MM causes the generated CryptX.so to export all manner of symbols that nothing needs and that can potentially conflict with other libraries. (The conflicts appear to happen only when Perl is embedded such that the conflicting library is loaded outside of Perl, e.g., mod_perl or perlcc.)

I’m seeing this kind of thing in other XS libraries, too, e.g., CBOR::Free.

It seems like, since only Perl loads these libraries, the only things exported should be the actual XS interface.

There are two potential fixes; we’re not sure which is ultimately best:

• Add -Wl,--exclude-libs,ALL
• Add -fvisibility=hidden

[FGasper]

+  if ($^O ne 'MSWin32' && $Config{ld} =~ /gcc|g\+\+/) {
+     push @EUMM_INC_LIB, (LDDLFLAGS => "$Config{lddlflags} -Wl,--exclude-libs,ALL");
+  }

^^ That’s the fix for CryptX.

Would someone more knowledgeable than myself about all of this be willing to comment on the feasibility of applying this to XS modules in general?

[FGasper]

Of note:

-rwxr-xr-x 1 root root 1193592 Dec 31 16:59 whatever.exclude-libs
-rwxr-xr-x 1 root root 1399704 Dec 30 23:59 whatever.stock.cryptx

NB: This reduced CryptX.so’s size by about 15%.

[Leont]

We actually do exactly this sort of thing on a few operating systems (windows, aix, vms, os2). Most of the infrastructure is already there, we just need to

1. Add an export list writer for GCC on ELF
2. Actually use it (and possible detect if current gcc/clang supports it)
3. Have a way to enable/disable it?

[craigberry]

We actually do exactly this sort of thing on a few operating systems (windows, aix, vms, os2). Most of the infrastructure is already there, we just need to

1. Add an export list writer for GCC on ELF
2. Actually use it (and possible detect if current gcc/clang supports it)
3. Have a way to enable/disable it?

@rafl had a go at doing this to core some time ago for GNU ld on ELF platforms:

https://github.com/Perl/perl5/commits/rafl/ld_export

This never got merged, and I think the difficulty was that it depends on your linker, not your compiler, and your linker may have come with your compiler, or it may have come with the OS, and there are a variety of different ways of enforcing strict linking, so it's pretty intimidating to try to do this portably.

For extension building, item 2 above is just as difficult as it would be in core, but item 1 is easier. At least as far as I can remember the symbol "list" is really just boot_XXX where XXX is the name of the extension. DynaLoader takes care of the rest at run time.

From a recent build of blead on VMS, the Fcntl extension gets a "linker options file" called Fcntl.opt with the following contents:

SYMBOL_VECTOR=(boot_Fcntl=PROCEDURE)
[--.lib.auto.Fcntl]Fcntl.olb/Include=Fcntl
[--.lib.auto.Fcntl]Fcntl.olb/Library
[--]PerlShr.exe/Share

On Windows with MSVC, you get a symbol definition file called Fcntl.def that contains this:

LIBRARY "Fcntl"
EXPORTS
  boot_Fcntl

So identifying that one symbol you need to export is not the hard part. But portably deducing whether and how the linker can consume a symbol list and in what format sounds like work :-). Excluding symbols from external libraries is a little more narrow a problem and would still be helpful, but I don't know how universal --exclude-libs is across unixy linkers.

[Leont]

But portably deducing whether and how the linker can consume a symbol list and in what format sounds like work :-).

I think that adding support for gcc (and clang) is all we really need here, and we already have a gccversion configuration variable. The other compilers are either already supported or probably don't support this at all.

[monkburger]

Indeed. Having symbol table full of excessive bloat causes;

• Performance issues, GCC might not be able to inline/optimization of the functions, indirect function calls, etc
• Slower Application Startup Time
• Leaked internal APIs that shouldn't be exposed to other programs/shared objects
• And of course, Symbol Collisions.