FEAT Add pluggable tools to OpenAI Responses target - #2718
Behnam (behnam-o) wants to merge 2 commits into
Conversation
Introduce typed Python tools and provider-based discovery so the Responses target can advertise, execute, and continue conversations across tool calls. Add MCP Streamable HTTP and stdio providers, examples, and end-to-end coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep the upstream custom_functions execution path alongside first-class tools so the rebased feature does not remove the newly landed API. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
9630d44 to
d89b3d2
Compare
| """ | ||
| async with AsyncExitStack() as provider_stack: | ||
| for provider in self._tool_providers: | ||
| if isinstance(provider, _ScopedToolProvider): | ||
| await provider_stack.enter_async_context(provider.execution_scope_async()) | ||
| return await self._run_tool_call_loop_async(normalized_conversation=normalized_conversation) |
There was a problem hiding this comment.
The MCP session changes how exceptions reach pyrit_target_retry. If the model request raises RateLimitException, exiting the MCP SDK's task groups wraps it in nested ExceptionGroups. The retry decorator does not recognize those, so a transient rate-limit failure aborts instead of retrying whenever an MCP provider is configured, even before any tool executes.
Could we handle retries around each model request inside the provider scope? That would let the retry handler see the original exception and avoid rerunning completed tool calls when a later model request needs a retry.
| return { | ||
| "type": self.__class__.__name__, | ||
| "server_name": self._server_name, | ||
| "transport": self._server_config.type, | ||
| } |
There was a problem hiding this comment.
Could this identifier include the non-secret configuration that determines which MCP server is used? For example, two providers named notes, one pointing to https://one.example/mcp and the other to https://two.example/mcp, currently have identical identifiers and target hashes. The saved-attack target guard therefore accepts switching between them as if the target were unchanged.
There is a related gap in OpenAIResponseTarget._build_identifier: direct tools contribute only their names, so changing a tool's schema or description also leaves its identity unchanged. We should capture the tool definitions and a safe provider configuration identity, without storing credentials.
| "extra_body_parameters": self._extra_body_parameters, | ||
| "tools": sorted(tool.name for tool in self._tools), |
There was a problem hiding this comment.
Could we omit these fields, or use None, when no tools or providers are configured? Adding tools=[] and tool_providers=[] changes the identifier hash for existing targets even when the caller does not use either new option. AttackService._validate_target_match then rejects continuing a saved attack with "Target mismatch", although its target configuration has not changed.
The identifier hashing code already excludes None, so that would preserve existing hashes for callers who do not opt into the new functionality.
2 participants
Summary
Tooland@toolAPIs for async Python functionsOpenAIResponseTargetadvertise and execute direct or provider-discovered tools across multi-step Responses API conversationsWhy
PyRIT could previously process OpenAI function-call responses only through manually paired schemas and callbacks. This makes tool advertisement and execution a supported, pluggable target capability while keeping OpenAI protocol handling in the target and MCP as an optional provider.