Reasoning for and potential side effects of linking cudart_static?

[BwL1289]

As discussed here, implemented here, and documented here, my team and I are wondering what the reasoning was behind linking unconditionally linking cudart_static in setup.py in the newest version of cuda_bindings?

We've read through the full conversation and are still unclear why it's needed.

Potentially more importantly, could there be any unintended side effects as cuda_bindings may also load libcudart dynamically at runtime?

The reason for asking is due to trying to keep the same cuda libraries consistently linked statically or dynamically across all libraries (for example, torch mixes linking static and dynamic cuda libraries and we want to ensure we're always linking the cuda libraries the same way).

[leofang]

You missed #100 which explained why we are doing this 🙂

[BwL1289]

Ah Thank you! And sorry for the noise.

Do you know if there could there be any unintended side effects as cuda_bindings may also load libcudart at runtime?

[BwL1289]

@leofang are there are unintended side effects of linking cudart_static at buildtime and loading cudart at runtime?

[leofang]

@BwL1289 In cuda.bindings, the only API that could load libcudart is runtime.getLocalRuntimeVersion(), which is a Python-only API that we added to check the version of the loaded libcudart (if any). Other APIs interact with the driver or static cudart directly.

In fact, the name "CUDA Runtime" is a misnomer. It is not an intelligent run-time system as you might have imagined by name. Nothing fancy is done there. The best way to think about it is "(slightly more user-friendly) C bindings" to driver APIs. The only thing that's fancy here is that it internally loads the driver (libcuda) and fetches the correct symbols based on the loaded driver version in a CUDA-minor-version-compatible manner. In fact, statically linking to cudart is the recommended practice:

• it's NVCC's default
• it is slightly more performant (unfortunately I can't say more other than what the linked comment says)
• it is what all CUDA libraries do (cuBLAS, cuFFT, ...)
• it allows your library/application to support CUDA minor version compatibility (with some asterisks)

So you should not expect any issues there. It's been done like this for many years.

[BwL1289]

@leofang sorry I missed this, and thanks. Just to clarify, why is it necessary to load the libcudart version if cuda.bindings is only linking statically?

To be clear, we prefer to link statically and were surprised to learn torch changed the default from linking statically to dynamically a few years ago. And I know cuda.bindings is not responsible for torch and vice versa. Their decision did make life more difficult for us...