Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lib vs lib64 in containers #120

Open
jfpanisset opened this issue Jun 28, 2021 · 5 comments
Open

lib vs lib64 in containers #120

jfpanisset opened this issue Jun 28, 2021 · 5 comments

Comments

@jfpanisset
Copy link
Contributor

Looking at the ci-vfxall image, it seems that most of the resulting DSOs end up in /usr/local/lib
but a few end up in /usr/local/lib64 (OpenGL, OIIO, Ptext, OSL). On CentOS 7 systems:

32 bit DSOs: /lib -> /usr/lib
64 bit DSOs: /lib64 -> /usr/lib64

Since these containers are 64 bit only, would it make sense to have all DSOs land in /usr/local/lib64 ? Could always have a backwards compatibility link from /usr/local/lib to /usr/local/lib64

On the other hand systemd defines /lib64 as a "backwards compatibility symlink":

https://www.freedesktop.org/software/systemd/man/file-hierarchy.html#Compatibility%20Symlinks

so it's unclear what's the correct approach, but since for now these containers are somewhat CentOS 7 centric, I would suggest following the lead of CentOS 7, and having all DSOs land in /usr/local/lib64
@aloysbaillet
Copy link
Contributor

Hi JF,

Yep I noticed that inconsistency as well, but I'm basically just getting what cmake decided to do at install time... Some packages are following centos' desire to put libs in lib64, and others just ignore that. And taking the time to post-process all package install procedures would take a bit of effort, and could break some packages in subtle ways (e.g. USD installs are not relocatable as far as I know, so moving the dso files after install could break things...).
That said if you know of a cool cmake trick to do this I'd be happy to try :)

Cheers,

Aloys

@jfpanisset
Copy link
Contributor Author

For the specific case of OpenImageIO, the CMake code that decides where to install the libraries seems to come from (or at least be related to):

https://github.com/OpenImageIO/oiio/blob/master/src/build-scripts/OpenImageIO.pc.in

where libdir is derived from @CMAKE_INSTALL_LIBDIR@ which claims to default to lib, but somehow here ends up being lib64

I'll try to experiment with build_oiio.sh and see why it ends up installing in lib64 instead of lib.

@lgritz
Copy link
Contributor

lgritz commented Jun 29, 2021

The .pc.in is just a template for how it generates a pkgconfig file describing where things were put by the install. It doesn't determine any of the logic of where things are put.

@lgritz
Copy link
Contributor

lgritz commented Jun 29, 2021

I believe this is the key to understanding: https://cmake.org/cmake/help/latest/module/GNUInstallDirs.html

jfpanisset added a commit to jfpanisset/aswf-docker that referenced this issue Jul 1, 2024
@jfpanisset
Midlife update to VFX 2024 build containers
34d2dbe 
- blosc moved to base1
- rework Conan packages to use CMakeToolChain, conandata.yml
- Conan packages mostly install to lib64 instead of lib (partial AcademySoftwareFoundation#120)
- libdeflate in base image for OpenEXR
- oiio build container (AcademySoftwareFoundation#173)
- Conan packages preserve DSO symlinks when installed (AcademySoftwareFoundation#194)
- Conan packages don't overwrite each other's installed license files
- Break circular dependency between OCIO and OIIO by compiling OCIO utils without OIIO (AcademySoftwareFoundation#54)
- OpenEXR and Imath built as Conan packages only
- MaterialX 1.38.10 (was 1.38.8)
- Imath 3.1.11 (was 3.1.10)
- OpenEXR 3.2.4 (was 3.2.2)
- OpenImageIO 2.5.12.0 (was 2.5.8.0)
- OpenShadingLanguage 1.13.10.0 (was 1.13.6.1)
- OpenTimelineIO 0.17.0 (was 0.15)
- Conan 1.63 (was 1.62)
- CMake 3.27.9 (was 3.27.8)
- Clang 16.0.6 (was 16.0.4), 17.0.6 (was 17.0.1)
- pybind11 2.12.0 (was 2.11.1)
- Python 3.11.9 (was 3.11.8)
- USD 24.05 (was 23.11)

In more details:

ci-oiio/Dockerfile
ci-oiio/README.md
ci-oiio/image.yaml

Dedicated build container for OpenImageIO

packages/common/Dockerfile

Possibly due to a Conan update, llvm was no longer getting
GCC_INSTALL_PREFIX when building as a Conan package, so it wouldn't
find the internal / C++ headers from gcc-toolkit. Now it does.

Also make sure to pass command line options to `conan create` before
the recipe path and package spec.

packages/conan/recipes/...

- use conandata.yml to download, verify SHA256 hashes and apply patches for
all Conan packages
- add some missing SPDR headers (work in progress)
- modernize recipes based on conan center index, use CMakeToolchain

packages/vfx1/Dockerfile

Build OCIO first and OIIO second and break circular dependency by not using OIIO for OCIO utilities

packages/vfx2/Dockerfile

blosc is now built as a Conan package, part of the `base1` package group.

scripts/common/install_yumpackages.sh

Add libdeflate dev package for OpenEXR

scripts/vfx/build_ocio.sh

Break circular dependency with OIIO

scripts/vfx/build_osl.sh

Update download URL to ASWF GitHub org

Signed-off-by: Jean-Francois Panisset <[email protected]>
@jfpanisset jfpanisset mentioned this issue Jul 1, 2024
Merged
jfpanisset added a commit that referenced this issue Jul 14, 2024
@jfpanisset
Midlife update to VFX 2024 build containers
ddd9e44 
- blosc moved to base1
- rework Conan packages to use CMakeToolChain, conandata.yml
- Conan packages mostly install to lib64 instead of lib (partial #120)
- libdeflate in base image for OpenEXR
- oiio build container (#173)
- Conan packages preserve DSO symlinks when installed (#194)
- Conan packages don't overwrite each other's installed license files
- Break circular dependency between OCIO and OIIO by compiling OCIO utils without OIIO (#54)
- OpenEXR and Imath built as Conan packages only
- MaterialX 1.38.10 (was 1.38.8)
- Imath 3.1.11 (was 3.1.10)
- OpenEXR 3.2.4 (was 3.2.2)
- OpenImageIO 2.5.12.0 (was 2.5.8.0)
- OpenShadingLanguage 1.13.10.0 (was 1.13.6.1)
- OpenTimelineIO 0.17.0 (was 0.15)
- Conan 1.63 (was 1.62)
- CMake 3.27.9 (was 3.27.8)
- Clang 16.0.6 (was 16.0.4), 17.0.6 (was 17.0.1)
- pybind11 2.12.0 (was 2.11.1)
- Python 3.11.9 (was 3.11.8)
- USD 24.05 (was 23.11)

In more details:

ci-oiio/Dockerfile
ci-oiio/README.md
ci-oiio/image.yaml

Dedicated build container for OpenImageIO

packages/common/Dockerfile

Possibly due to a Conan update, llvm was no longer getting
GCC_INSTALL_PREFIX when building as a Conan package, so it wouldn't
find the internal / C++ headers from gcc-toolkit. Now it does.

Also make sure to pass command line options to `conan create` before
the recipe path and package spec.

packages/conan/recipes/...

- use conandata.yml to download, verify SHA256 hashes and apply patches for
all Conan packages
- add some missing SPDR headers (work in progress)
- modernize recipes based on conan center index, use CMakeToolchain

packages/vfx1/Dockerfile

Build OCIO first and OIIO second and break circular dependency by not using OIIO for OCIO utilities

packages/vfx2/Dockerfile

blosc is now built as a Conan package, part of the `base1` package group.

scripts/common/install_yumpackages.sh

Add libdeflate dev package for OpenEXR

scripts/vfx/build_ocio.sh

Break circular dependency with OIIO

scripts/vfx/build_osl.sh

Update download URL to ASWF GitHub org

Signed-off-by: Jean-Francois Panisset <[email protected]>
@jfpanisset
Copy link
Contributor Author

For USD, looks at setting PXR_ALL_LIBS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@lgritz @aloysbaillet @jfpanisset