gh-119609: Add PyUnicode_Export() and PyUnicode_Import() functions

[vstinner]

• Issue: [C API] Add PyUnicode_Export() and PyUnicode_Import() functions #119609

---

📚 Documentation preview 📚: https://cpython-previews--119610.org.readthedocs.build/

[vstinner]

Docs build failed because of #119607

[vstinner]

Docs build failed because of #119607

PR rebased on top of this fix.

[zooba]

Petr indicated on the issue that he's got Thoughts and will write them up this week.

[vstinner]

@encukou: Please review the updated PR.

@encukou: I prefer to make sure that the exported string ends with a NUL character, rather than making sure that it's not the case. It's convenient and cheap.

I rebased the PR on the main branch, I fixed merge conflicts, I updated the doc for new function names, I included some of Petr's suggestions. I marked the PR as ready for review (it's no longer a draft).

[encukou]

Looks good to me!
(I don't know about the other C API WG members.)

[vstinner]

@davidism: Does the proposed API solve your MarkupSafe use case?

[vstinner]

I created Add PyUnicode_Export() and PyUnicode_Import() to the limited C API issue in the C API WG Decisions project.

[davidism]

I really appreciate you thinking about this after talking at PyCon! From what I can tell (I'm not very familiar with C), the MarkupSafe code would mostly remain the same, but use the two new abi3 functions instead of the existing functions? Perhaps you could show a brief example or a high level explanation of what I would change? https://github.com/pallets/markupsafe/blob/d12057361ad75c4569e2e61712c234acc69d5d0b/src/markupsafe/_speedups.c

[vstinner]

@davidism:

I really appreciate you thinking about this after talking at PyCon!

You're welcome.

From what I can tell (I'm not very familiar with C), the MarkupSafe code would mostly remain the same, but use the two new abi3 functions instead of the existing functions? Perhaps you could show a brief example or a high level explanation of what I would change?

I created a PR to show how these functions can be used: pallets/markupsafe#451

The stable ABI is less efficient since it requires to allocate a UCS1/UCS2/UCS4 buffer first, write into the buffer, and only then create a Python str object from this buffer. That's because PyUnicode_New() is excluded from the stable ABI.