-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Add ValidationOptions for lenient output schema validation #1260
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add ValidationOptions for lenient output schema validation #1260
Conversation
3943c45
to
26c6806
Compare
26c6806
to
7722e38
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @lorenss-m, thanks for sending this!
Seems reasonable to add this option given the spec only says clients "should" validate structured results against the output schema.
I'm curious tho, which examples of non-conforming outputs did you find / how bad were they?
cc/ @bhosmer-ant since authored initial support in #993
) | ||
else: | ||
try: | ||
validate(result.structuredContent, output_schema) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What error would None produce here, do we need the two branches?
class ValidationOptions(BaseModel): | ||
"""Options for controlling validation behavior in MCP client sessions.""" | ||
|
||
strict_output_validation: bool = Field( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wondering if we wanna make this sound more explicitly about tool call output, as there's also validation happening around elicitation (https://modelcontextprotocol.io/specification/2025-06-18/client/elicitation#security-considerations)
Maybe something like validate_structured_outputs
(no sure we need strict in the name, if we don't raise an exception the output isn't really validated, warnings being often ignored)
validate(result.structuredContent, output_schema) | ||
except ValidationError as e: | ||
if self._validation_options.strict_output_validation: | ||
raise RuntimeError(f"Invalid structured content returned by tool {name}: {e}") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: might wanna raise ... from e
Motivation and Context
Currently, MCP clients raise
RuntimeError
when tools don't return structured content matching theiroutputSchema
. This breaks integrations with servers that have schema/implementation mismatches.This PR adds optional lenient validation that converts these errors to warnings, allowing clients to continue operating with imperfect servers.
How Has This Been Tested?
tests/client/test_validation_options.py
Breaking Changes
None. Default behavior unchanged. Lenient validation is opt-in:
Types of changes
Checklist
Additional context
ValidationOptions
class following existing patterns (e.g.,TransportSecuritySettings
)ClientSession.call_tool()
to check options before raising validation errorsstructured_output_lowlevel.py
by creating proper package structure