Inconsitency in APM trace spans relating LangChain integration

[Peilun-Li]

Environment

• dd-trace-py: 2.16.4
• langchain-core: 0.2.34
• langchain-community: 0.2.1
• langchain-openai: 0.1.8

Reproduce

A standard langchain AgentExecutor with a retriever tool: https://python.langchain.com/docs/how_to/agent_executor/#retriever and ainvoke the agent with a question to trigger the retriever tool.

Expectation

APM trace span for the request would show three sections:

1. Initial call to LLM (LLM returns desire to usage the retriever tool)
2. Call retriever/vector store
3. Second call to LLM (LLM generates answer with retrieved context)

Actual behavior

Inconsistent behavior of APM trace span among four test calls (all same output and all triggered tool usage)

First call only shows 1. and 2. above (2. is inferred through request)

Second call only shows 1.

Third call only shows 1. and 2., with 1. incomplete (only 2 levels compared to 4 levels in the first call)

Fourth call only shows 1. and 3., with 3. incomplete (expect the same 4 levels for OpenAI calls but only get 2 levels)

To summarize, it looks like while we do support inject spans for all the expected three sections, they are quite unstable and inconsistent across requests. Could it be due to some setup from our application side, or a ddtrace library issue, or some others? Appreciate any help/insight!