-
Notifications
You must be signed in to change notification settings - Fork 476
Description
Tracer Version(s)
3.19.0, 4.0.0
Python Version(s)
Python 3.11.13
Pip Version(s)
pip 24.3.1
Bug Report
When DD_PROFILING_ENABLED=true, applications using Neo4j's AsyncGraphDatabase fail to start with a TypeError on Azure v6-series VMs only. The same configuration works correctly on v5-series VMs. Regular ddtrace APM tracing (without profiling) works fine on both VM generations.
Reproduction Pattern (Confirmed Across Multiple Regions)
| Region | VM SKU | DD_PROFILING_ENABLED | Result |
|---|---|---|---|
| WestUS2 | Standard_E16ds_v5 (Cascade Lake) | true | ✅ Works |
| WestUS2 | Standard_E16ds_v6 (Ice Lake) | true | ❌ Fails |
| WestUS2 | Standard_E16ds_v6 (Ice Lake) | false | ✅ Works |
| WestEU | Standard_E8ds_v6 (Ice Lake) | false | ✅ Works |
| WestEU | Standard_E8ds_v6 (Ice Lake) | true | ❌ Fails |
Root Cause
When profiling is enabled, ddtrace registers an additional hook on the asyncio module that wraps asyncio.Lock with _LockAllocatorWrapper, changing its __init__ signature. When Neo4j's async driver attempts to subclass it with class AsyncRLock(asyncio.Lock):, the inheritance fails due to the signature mismatch.
Critical finding: This issue only manifests on Azure v6-series VMs (Intel Ice Lake processors), not on v5-series VMs (Intel Cascade Lake processors), suggesting a timing or initialization difference in how Python modules are loaded across CPU architectures.
Reproduction Code
# This fails ONLY when DD_PROFILING_ENABLED=true on Azure v6 VMs
from neo4j import AsyncGraphDatabase
# The import fails at Neo4j driver initialization:
# File "neo4j/_async_compat/concurrency.py", line 49
# class AsyncRLock(asyncio.Lock):
# TypeError: _LockAllocatorWrapper.__init__() takes 2 positional arguments but 4 were givenDeployment:
- Azure Kubernetes Service
- Datadog Admission Controller injection
- Environment variable:
DD_PROFILING_ENABLED=true
Error Logs
DD_TRACE_DEBUG=true output shows:
Registering hook '<function _ProfilerInstance.__post_init__.<locals>.<lambda>' on module 'asyncio'
Calling hook on already imported module 'asyncio'
Profiling StackCollector starting
Traceback (most recent call last):
File "/deps/agents/agents/cos_chatbot/chat_utils/graph_data.py", line 6, in <module>
from neo4j import AsyncGraphDatabase
File "/usr/local/lib/python3.11/site-packages/neo4j/__init__.py", line 91, in <module>
from neo4j._async.driver import (
File "/usr/local/lib/python3.11/site-packages/neo4j/_async/driver.py", line 37, in <module>
from .._async_compat.concurrency import (
File "/usr/local/lib/python3.11/site-packages/neo4j/_async_compat/concurrency.py", line 49, in <module>
class AsyncRLock(asyncio.Lock):
TypeError: _LockAllocatorWrapper.__init__() takes 2 positional arguments but 4 were given
Libraries in Use
neo4j==5.26.0
ddtrace==3.19.0
langchain-core==0.3.25
langchain-community==0.3.13
langgraph==0.2.59
Operating System
Linux (Azure Kubernetes Service)
- ✅ Works: Standard_E16ds_v5 VMs (Intel Cascade Lake)
- ❌ Fails: Standard_E16ds_v6 VMs (Intel Ice Lake)
- ❌ Fails: Standard_E8ds_v6 VMs (Intel Ice Lake)
Workaround: Set DD_PROFILING_ENABLED=false. Regular APM tracing continues to work.