Python 3.13 support

[iherasymenko]

When trying to install ddtrace-py with the latest Python 3.13 RC builds, the installation fails with the following error.

building 'ddtrace.profiling.collector._memalloc' extension
creating build/temp.linux-aarch64-cpython-313/ddtrace/profiling/collector
gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/tmp/tmperfe434l/.venv/include -I/usr/local/include/python3.13 -c ddtrace/profiling/collector/_memalloc.c -o build/temp.linux-aarch64-cpython-313/ddtrace/profiling/collector/_memalloc.o
gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/tmp/tmperfe434l/.venv/include -I/usr/local/include/python3.13 -c ddtrace/profiling/collector/_memalloc_heap.c -o build/temp.linux-aarch64-cpython-313/ddtrace/profiling/collector/_memalloc_heap.o
gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/tmp/tmperfe434l/.venv/include -I/usr/local/include/python3.13 -c ddtrace/profiling/collector/_memalloc_tb.c -o build/temp.linux-aarch64-cpython-313/ddtrace/profiling/collector/_memalloc_tb.o
gcc -shared build/temp.linux-aarch64-cpython-313/ddtrace/profiling/collector/_memalloc.o build/temp.linux-aarch64-cpython-313/ddtrace/profiling/collector/_memalloc_heap.o build/temp.linux-aarch64-cpython-313/ddtrace/profiling/collector/_memalloc_tb.o -L/usr/local/lib -o build/lib.linux-aarch64-cpython-313/ddtrace/profiling/collector/_memalloc.cpython-313-aarch64-linux-gnu.so
building 'ddtrace.internal._threads' extension
creating build/temp.linux-aarch64-cpython-313/ddtrace/internal
g++ -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/tmp/tmperfe434l/.venv/include -I/usr/local/include/python3.13 -c ddtrace/internal/_threads.cpp -o build/temp.linux-aarch64-cpython-313/ddtrace/internal/_threads.o -std=c++17 -Wall -Wextra
ddtrace/internal/_threads.cpp: In constructor ‘GILGuard::GILGuard()’:
ddtrace/internal/_threads.cpp:23:14: error: ‘_Py_IsFinalizing’ was not declared in this scope; did you mean ‘Py_IsFinalizing’?
   23 |         if (!_Py_IsFinalizing())
      |              ^~~~~~~~~~~~~~~~
      |              Py_IsFinalizing
ddtrace/internal/_threads.cpp: In constructor ‘AllowThreads::AllowThreads()’:
ddtrace/internal/_threads.cpp:45:14: error: ‘_Py_IsFinalizing’ was not declared in this scope; did you mean ‘Py_IsFinalizing’?
   45 |         if (!_Py_IsFinalizing())
      |              ^~~~~~~~~~~~~~~~
      |              Py_IsFinalizing
ddtrace/internal/_threads.cpp: In destructor ‘AllowThreads::~AllowThreads()’:
ddtrace/internal/_threads.cpp:50:14: error: ‘_Py_IsFinalizing’ was not declared in this scope; did you mean ‘Py_IsFinalizing’?
   50 |         if (!_Py_IsFinalizing())
      |              ^~~~~~~~~~~~~~~~
      |              Py_IsFinalizing

[sanchda]

@iherasymenko thank you for the report! We're still addressing 3.13 support for the library. For the time being, I think it is safe to treat 3.13 as unsupported by ddtrace. At the current time we do not have a precise timeline (sorry about that).

I don't really have a specific workaround to suggest here, since we have a lot of native code that needs to be made away of changes to the runtime API in 3.13.

[sanchda]

Oops, just realized I mischaracterized this as a profiler issue, but I see that the compilation site is actually coming from a different component. Re-routing this issue, but unfortunately I think my overall message is still accurate.

[sanchda]

@emmettbutler directing to you in case you have anything more optimistic to say on the topic of 3.13 support.

[leandrodamascena]

Hi, do you have any updates on this issue?! I really want to use ddtrace with Python 3.13.

I was reading through the Python 3.13 API changes and the necessary changes to make ddtrace aware of Python 3.13 and I see some important changes.

_Py_IsFinalizing Part of the Stable ABI since version 3.13.

_Py_IsFinalizing is part of the Stable ABI and now its Py_IsFinalizing, so, the following code (we can optimize, of course) can keep backward compatibility (https://github.com/DataDog/dd-trace-py/blob/main/ddtrace/internal/_threads.cpp) and add support for Python 3.13

#if PY_VERSION_HEX < 0x30d0000
    if (!_Py_IsFinalizing()) {
#else
    if (!Py_IsFinalizing()) {
#endif

Py_BUILD_CORE must be define when importing internal

Accessing some internal headers now requires defining Py_BUILD_CORE. However, if I'm not reading wrong, this define is only meant to be used when building CPython itself. So, I think adding Py_BUILD_CORE define (+ python version check) on top of the file https://github.com/DataDog/dd-trace-py/blob/b849a2a1197265a43a4960f49d302f4eccaa3470/ddtrace/appsec/_iast/_stacktrace.c can fix the problem.

Building datadog/profiling/stack_v2

I see some errors like fatal error: 'internal/pycore_opcode.h' file not found, but I don't know if this is a problem when cmake is linking the headers or not. I'm still debugging.

I see some other debugs when compiling it locally, but I don't know if that's my environment or not. I can't find in the repository any guidance to set up my local environment correctly.

Please let me know if I can help in any other way to add this support in a timely manner.

[P403n1x87]

@leandrodamascena thanks for your interest in getting ddtrace to work with CPython 3.13. We are currently working towards this, and you have already identified many of the places where support needs to be extended. Besides that, we also need a new release of the bytecode library, which is on its way. We currently don't have a timescale for when all the work will be done, but we are confident we can get there within a few weeks 🤞

[leandrodamascena]

@leandrodamascena thanks for your interest in getting ddtrace to work with CPython 3.13. We are currently working towards this, and you have already identified many of the places where support needs to be extended. Besides that, we also need a new release of the bytecode library, which is on its way. We currently don't have a timescale for when all the work will be done, but we are confident we can get there within a few weeks 🤞

Thanks for letting me know @P403n1x87! Feel free to reach out if you need someone to do some testing with Python 3.13

[evanstjabadi]

@P403n1x87, Nice updated shared.

Besides that, we also need a new release of the bytecode library, which is MatthieuDartiailh/bytecode#146.

It looks like bytecode is now cleared for Python 3.13 🎉

[dima-aronov-dev]

Looking forward for the python 3.13 support :)