cmake: make possible to enable LTO when compiling dll games

[illwieckz]

The -fPIE and -fpie flags should not be used when building a shared library. For some unknown reasons that works with GCC and Clang until we enable LTO.

This patch makes CMake uses -fpie when linking executables, and -fpic when linking libraries.

Because common object files from shared source between engine and game cane be built once for both engine executable and game dll, all objects are built with -fPIC if building dll games is enabled, or built with -fPIE otherwise.

It is then not possible to build an engine executable with -fPIE when building dll games in the same build folder. To build engine executable with -fPIE and dll games with -fPIE in the same build folder, we would have to implement a sub-cmake like we do for NaCl, losing the ability to reuse object files.

When building a release we not only don't build the dll game but also build the engine separately with any game disabled, so this should change nothing to the way releases are built.

For information, here is the kind of linking error I get with GCC when enabling both HARDENING (and then PIE) and LTO with dll games:

[100%] Linking CXX shared module cgame-native-dll.so
/usr/bin/ld: /tmp/ccg6Lf14.ltrans22.ltrans.o: warning: relocation against `_ZTIi@@CXXABI_1.3' in read-only section `.text'
/usr/bin/ld: /tmp/ccg6Lf14.ltrans0.ltrans.o: relocation R_X86_64_PC32 against symbol `_ZTVNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEEE@@GLIBCXX_3.4.21' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status

[slipher]

Because common object files from shared source between engine and game cane be built once for both engine executable and game dll, all objects are built with -fPIC if building dll games is enabled, or built with -fPIE otherwise.

The ASLR for DLLs is prioritized over ASLR for executables? That’s bad. DLLs are only for debugging and no one should give a shit about using ASLR for them. It would make more sense to just force BUILD_GAME_DLL to off when LTO is requested.

[illwieckz]

Building a dll game is only meaningful for debugging or profiling anyway, and I want to be able to profile binaries built with LTO enabled. Why caring about engine debugging builds not being like release ones?

[slipher]

It's a bad and suprising-to-the-user idea to make the engine build behave differently depending on whether the DLL build is enabled or not. For builds intended to be use as "release" it's surely very common to have the DLL builds enabled. For a user who is not intimately familiar with Unvanquished development, the obvious thing to do is checkout out Unvanquished and then build it. Not check out Daemon or futz around with disabling targets.

[illwieckz]

I don't get why the default game build is dll and not nexe, to begin with. The dll game is never meant for release..

[slipher]

I don't get why the default game build is dll and not nexe, to begin with. The dll game is never meant for release..

Well, building any gamelogic at all is something that is for developers/maintainers, not users. But catering purely to end users would be too perverse as the Unvanquished repo wouldn't build any of its own code by default :) DLL is the most commonly used dev workflow, so that is the default.

[illwieckz]

If dll is a dev workflow I don't think we should worry about the engine not being a release build…

But anyway, I did that instead:

• cmake: build dll game with sub cmake to use different compilation flags Unvanquished/Unvanquished#3270

It builds the dll game in a cmake subproject so we can use different compilation and linker flags than when building the engine.