Add PyMutex, PyMutex_Lock, PyMutex_Unlock

[Rogdham]

Hello, I am doing a backport of code using PyMutex/PyMutex_Lock/PyMutex_Unlock. Could it be possible to add them to this repository?

References:

• Python documentation
• gh-117511 issue on CPython

[ngoldbaum]

I think backporting PyMutex would be nontrivial, since it relies on integration with mimalloc AIUI.

Instead, you could follow what we did in NumPy and explicitly wrap PyThread_type_lock and PyMutex in a compatibility macro, or similar. numpy/numpy#27011

I'm not sure if it makes sense to put something like that in pythoncapi-compat.

[Rogdham]

I was hopping that it would have been possible to have a generic way to implement it here, maybe using PyThread_type_lock internally.

But after playing for a while with your suggestion (which was also Emma's), I'm starting to wonder if it's possible at all, as PyMutex API lacks the init & free methods.

But my knowledge about both C and the CPython C API is limited, so I would be happy if someone proves me wrong!

In the meanwhile, I will work following the suggestion previously mentioned. Thanks!

[ngoldbaum]

PyMutex doesn't need to be heap-allocated, so you can leave the allocate and free functions as stubs. The docs for PyMutex say you need to initialize like it like e.g.

PyMutex mut = {0};

But that's the only special thing you need to handle.

[Rogdham]

Sorry, I was not clear at all in my last message.

I was thinking if it was possible, as a compatibility layer to PyMutex, to implement PyMutex using PyThread_type_lock for older versions of Python not having PyMutex.

Like you mentioned, since the initialization of PyMutex in code that would use it does not call any function, I don't see where the call to PyThread_allocate_lock can be hooked up in the compatibility layer.

Likewise for calling PyThread_free_lock.

[ngoldbaum]

Couldn't the compat function just not do anything to allocate and free the object in the PyMutex branch that 3.13 and newer use?

For example, in NpyString_free_allocator in NumPy, there's an ifdef so that on older python versions we free the PyThread_type_lock mutex that hangs off the allocator but on newer versions we do nothing because PyMutex doesn't need to be free'd.

https://github.com/numpy/numpy/blob/3f1b4571cedefe505a281a2ed5c865f1e6e35e63/numpy/_core/src/multiarray/stringdtype/static_string.c#L269-L284

There's a similar ifdef in NpyString_new_allocator. You can grep static_string.c in NumPy for PY_VERSION_HEX to find all the version-dependent code. It turns out that StringDType was the only place where we were using PyThread_type_lock before we added free-threaded support and we're only using PyMutex elsewhere in code paths that are only unsafe under the free-threaded build.

Does that make sense?

[vstinner]

PyMutex was added to Python 3.13 and it's part of the public C API since Python 3.13. You don't need pythoncapi-compat on Python 3.13 and newer.

Backporting the whole feature to Python 3.12 would be too much work (add too much code, complicated to maintain).

I suggest closing this issue. You have to add two different code paths depending on the Python version: PyMutex for Python 3.13 and newer, or PyThread_allocate_lock() on Python 3.12 and older.g

By the way, I'm working on python/cpython#134747 : "Change PyThread_allocate_lock() implementation to PyMutex" on Python 3.15 :-)

[Rogdham]

I understand that backporting PyMutex for Python 3.12 and below is not something you want to support in this repository. Instead, users should make the necessary changes in their code to use either PyMutex or PyThread_allocate_lock based on the Python version.

Thank you for the pointers!