Skip to content

[BUG]: DD_PROFILING_ENABLED causes TypeError with Neo4j AsyncGraphDatabase on Azure v6 VMs (works on v5) #15558

@MattMencel

Description

@MattMencel

Tracer Version(s)

3.19.0, 4.0.0

Python Version(s)

Python 3.11.13

Pip Version(s)

pip 24.3.1

Bug Report

When DD_PROFILING_ENABLED=true, applications using Neo4j's AsyncGraphDatabase fail to start with a TypeError on Azure v6-series VMs only. The same configuration works correctly on v5-series VMs. Regular ddtrace APM tracing (without profiling) works fine on both VM generations.

Reproduction Pattern (Confirmed Across Multiple Regions)

Region VM SKU DD_PROFILING_ENABLED Result
WestUS2 Standard_E16ds_v5 (Cascade Lake) true ✅ Works
WestUS2 Standard_E16ds_v6 (Ice Lake) true ❌ Fails
WestUS2 Standard_E16ds_v6 (Ice Lake) false ✅ Works
WestEU Standard_E8ds_v6 (Ice Lake) false ✅ Works
WestEU Standard_E8ds_v6 (Ice Lake) true ❌ Fails

Root Cause

When profiling is enabled, ddtrace registers an additional hook on the asyncio module that wraps asyncio.Lock with _LockAllocatorWrapper, changing its __init__ signature. When Neo4j's async driver attempts to subclass it with class AsyncRLock(asyncio.Lock):, the inheritance fails due to the signature mismatch.

Critical finding: This issue only manifests on Azure v6-series VMs (Intel Ice Lake processors), not on v5-series VMs (Intel Cascade Lake processors), suggesting a timing or initialization difference in how Python modules are loaded across CPU architectures.

Reproduction Code

# This fails ONLY when DD_PROFILING_ENABLED=true on Azure v6 VMs
from neo4j import AsyncGraphDatabase

# The import fails at Neo4j driver initialization:
# File "neo4j/_async_compat/concurrency.py", line 49
#     class AsyncRLock(asyncio.Lock):
# TypeError: _LockAllocatorWrapper.__init__() takes 2 positional arguments but 4 were given

Deployment:

  • Azure Kubernetes Service
  • Datadog Admission Controller injection
  • Environment variable: DD_PROFILING_ENABLED=true

Error Logs

DD_TRACE_DEBUG=true output shows:

Registering hook '<function _ProfilerInstance.__post_init__.<locals>.<lambda>' on module 'asyncio'
Calling hook on already imported module 'asyncio'
Profiling StackCollector starting

Traceback (most recent call last):
  File "/deps/agents/agents/cos_chatbot/chat_utils/graph_data.py", line 6, in <module>
    from neo4j import AsyncGraphDatabase
  File "/usr/local/lib/python3.11/site-packages/neo4j/__init__.py", line 91, in <module>
    from neo4j._async.driver import (
  File "/usr/local/lib/python3.11/site-packages/neo4j/_async/driver.py", line 37, in <module>
    from .._async_compat.concurrency import (
  File "/usr/local/lib/python3.11/site-packages/neo4j/_async_compat/concurrency.py", line 49, in <module>
    class AsyncRLock(asyncio.Lock):
TypeError: _LockAllocatorWrapper.__init__() takes 2 positional arguments but 4 were given

Libraries in Use

neo4j==5.26.0
ddtrace==3.19.0
langchain-core==0.3.25
langchain-community==0.3.13
langgraph==0.2.59

Operating System

Linux (Azure Kubernetes Service)

  • ✅ Works: Standard_E16ds_v5 VMs (Intel Cascade Lake)
  • ❌ Fails: Standard_E16ds_v6 VMs (Intel Ice Lake)
  • ❌ Fails: Standard_E8ds_v6 VMs (Intel Ice Lake)

Workaround: Set DD_PROFILING_ENABLED=false. Regular APM tracing continues to work.

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions