[BUG]: Unable to reliably invoke `nvjitlink.add_file` across threads

[cpcloud]

Is this a duplicate?

• I confirmed there appear to be no duplicate issues for this bug and that I agree to the Code of Conduct

Type of Bug

Runtime Error

Component

cuda.bindings

Describe the bug

This test occasionally fails when run with the GIL disabled:

    def test_add_file(arch, ptx_bytes, tmp_path):
        handle = nvjitlink.create(1, [f"-arch={arch}"])
        file_path = tmp_path / "test_file.cubin"
        file_path.write_bytes(ptx_bytes)
        nvjitlink.add_file(handle, nvjitlink.InputType.ANY, str(file_path))

with this traceback:

cuda_bindings/tests/test_nvjitlink.py:123:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cuda/bindings/nvjitlink.pyx:146: in cuda.bindings.nvjitlink.add_file
    ???
cuda/bindings/nvjitlink.pyx:162: in cuda.bindings.nvjitlink.add_file
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   cuda.bindings.nvjitlink.nvJitLinkError: ERROR_INVALID_INPUT (3)

How to Reproduce

python -Xgil=0 -m pytest --require-gil-disabled 'cuda_bindings/tests/test_nvjitlink.py::test_add_file[sm_75_6.4]' --iterations 30 --threads 10

Expected behavior

I'm not sure. Maybe this operation doesn't make sense across threads. If not, then a better error message might be nice.

That said, this API looks very much like it has global state based on the way it's being invoked (as a function, instead of as a method on handle), so maybe this does make sense.

Operating System

Linux

nvidia-smi output

I don't think nvidia-smi output is applicable here.

[kkraus14]

@cpcloud tmp_path is different per thread invoking the test, correct? If so, this is a bit surprising. If not, this is expected as writing the bytes to the file will clobber each other / race.

[cpcloud]

tmp_path is not different per thread, but should be relatively easy to adjust.