Skip to content

chore: Addressing blitz feedback#115

Open
jsonbailey wants to merge 14 commits intomainfrom
jb/aic-1664/blitz-feedback
Open

chore: Addressing blitz feedback#115
jsonbailey wants to merge 14 commits intomainfrom
jb/aic-1664/blitz-feedback

Conversation

@jsonbailey
Copy link
Copy Markdown
Contributor

@jsonbailey jsonbailey commented Mar 30, 2026

Note

Medium Risk
Changes tool binding/normalization and metrics attribution in the LangChain/LangGraph and OpenAI Agents runners, which can affect tool execution and telemetry emitted during agent/graph runs. Risk is mitigated by added integration-style tracking tests, but behavior depends on external SDKs (langgraph, openai-agents, OpenAI tools API).

Overview
Improves agent/graph tool wiring and telemetry across providers. LangChain removes structured-tool wrapping in favor of passing registry callables directly (build_tools), drops tool binding from create_langchain_model, and centralizes “last message output” extraction via extract_last_message_content used by both agent and LangGraph runners.

OpenAI provider updates tool support and tracking for openai-agents. Tool handling is refactored to accept either plain callables or native agents Tool instances (registry_value_to_agent_tool), sanitize agent names for SDK compatibility, and attribute $ld:ai:tool_call events by parsing RunResult.new_items. OpenAI model creation now normalizes LD tool definitions to the Chat Completions API shape (including native tool type mapping).

Packaging/tests: Adds optional dependency groups for langgraph and openai-agents, and introduces new integration-style tracking tests for LangGraph and OpenAI Agents to validate emitted LD events (tokens, latency, path, handoffs, tool calls).

Written by Cursor Bugbot for commit 51f3ba6. This will update automatically on new commits. Configure here.

jsonbailey and others added 14 commits March 30, 2026 11:23
Raw Python callables passed to bind_tools() lack the schema metadata
required to generate a valid OpenAI tools[].function spec, causing a
400 "Missing required parameter: 'tools[0].function'" error.

Wrapping each callable with StructuredTool.from_function(name=config_key)
also ensures the model response carries the config key as the tool name,
fixing a separate bug where function.__name__ was being tracked instead
of the LaunchDarkly config key.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_to_openai_name converted hyphens to underscores based on an assumed
restriction that doesn't exist — the OpenAI API allows hyphens in tool
and function names (^[a-zA-Z0-9_-]{1,64}$). Removing it simplifies the
tool name lookup and agent naming.

Also adds model=model.name to the Agent constructor so each node runs
with its configured model rather than the SDK default.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously, tool tracking relied on wrapping custom FunctionTools, which
meant native hosted tools (WebSearchTool, FileSearchTool, etc.) were
never tracked since they run server-side with no local callback.

Instead, parse result.new_items after the run completes. Each ToolCallItem
carries the originating agent (node attribution) and raw tool call data,
covering both custom and native tools without requiring local execution.

Also adds openai-agents as an optional dependency and dev dependency.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
langgraph is required to use LangGraphAgentGraphRunner but was not listed
as a dependency. Adding it as an optional extra (graph) and dev dependency,
consistent with how openai-agents is handled in the openai provider.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove FileSearchTool, CodeInterpreterTool, ImageGenerationTool from
  the native tool map; they require mandatory constructor args not
  available from the LD config and were already broken before.
- Move StructuredTool import inside the tools loop so it is only
  attempted when tools are present, fixing a test that mocks
  langchain_core without langchain_core.tools.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
main refactored _build_native_tool_map into openai_helper.py as
NATIVE_OPENAI_TOOLS and improved get_ai_usage_from_response to handle
RunResult objects. Merged with our changes by:

- Keeping only WebSearchTool in NATIVE_OPENAI_TOOLS (others require
  mandatory constructor args not available from config)
- Importing NATIVE_OPENAI_TOOLS, get_ai_usage_from_response, and
  get_tool_calls_from_run_items from the helper
- Retaining model=model.name in the Agent constructor
- Retaining result-based tool call tracking (no wrapper tracking)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jsonbailey jsonbailey marked this pull request as ready for review April 1, 2026 22:15
@jsonbailey jsonbailey requested a review from a team as a code owner April 1, 2026 22:15

# --- tools ---
agent_tools: List[Tool] = []
agent_tools: List[Any] = []
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be too annoying to type this to each tool? Basically Union[FunctionTool, WebSearchTool, ComputerUseTool....] They don't export a "base" tool unfortunately

Comment on lines +100 to +103
log.warning(
f"Tool '{name}' is defined in the AI config but was not found in "
"the tool registry; skipping."
)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: not addressing your PR here but should we re-think the strict linting rule for line length? Seems like we run into it more than just with my long prompts

Copy link
Copy Markdown
Contributor

@andrewklatzke andrewklatzke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Paid particular attention to the OpenAI path as I know langgraph is still being tweaked, left just one question

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants