Skip to content

Fix StructuredOutputValidationAdvisor dropping tool call responses#6588

Open
subhashpolisetti wants to merge 1 commit into
spring-projects:mainfrom
subhashpolisetti:fix-validation-advisor-tool-call-retry
Open

Fix StructuredOutputValidationAdvisor dropping tool call responses#6588
subhashpolisetti wants to merge 1 commit into
spring-projects:mainfrom
subhashpolisetti:fix-validation-advisor-tool-call-retry

Conversation

@subhashpolisetti

Copy link
Copy Markdown
Contributor

By default StructuredOutputValidationAdvisor runs at LOWEST_PRECEDENCE - 2000,
inside the ToolCallingAdvisor loop (HIGHEST_PRECEDENCE + 300). When structured
output validation is combined with tool calling and the model answers a validation
retry with a tool call, that response is silently discarded. The advisor re-invokes
the model with the same augmented prompt until maxRepeatAttempts is exhausted,
wasting the remaining retry budget on calls whose tool-call responses are thrown
away, and delaying the tool execution the model asked for.

Root cause

The retry loop exits only when validation succeeds or the attempt budget runs out.
Tool-call responses correctly skip the validation block (per the existing "We should
not validate tool call requests" comment), but nothing ended the loop on that branch,
so the failure flag stayed set and every tool-call round was treated as another
failed attempt. On the first attempt it worked only by accident, because
isValidationSuccess still held its initial true.

Fix

Break out of the retry loop when the model responds with a tool call, so the response
is handed back to the outer ToolCallingAdvisor for execution, consistent with the
first-attempt behavior. Validation still runs on every content response, and the
retry and augmentation logic is unchanged.

Testing

whenToolCallResponseFollowsValidationFailureThenReturnedForToolExecution fails on
the current implementation, where the chain is invoked a third time after the tool
call (expected: 2 but was: 3), and passes with the fix.
whenToolCallResponseOnFirstAttemptThenReturnedWithoutValidation pins the
already-correct first-attempt behavior.

Both tests are deterministic and provider-free, using DefaultAroundAdvisorChain with
a terminal advisor (the same harness as the existing tests). Existing tests are
unaffected, and ./mvnw -pl spring-ai-client-chat clean test passes (571 tests).

After a JSON validation failure, a retry round that responded with a
tool call request skipped validation but left the failure flag set, so
the retry loop discarded the response and re-invoked the model with
the same augmented request until the retry budget was exhausted. On
the first attempt the same response shape was returned correctly only
because the flag still held its initial value.

Break out of the retry loop when the model responds with a tool call,
so the response is handed back to the outer tool-calling advisor for
execution, consistent with the first-attempt behavior.

Signed-off-by: subhash polisetti <subhashr161347@gmail.com>
@subhashpolisetti subhashpolisetti force-pushed the fix-validation-advisor-tool-call-retry branch from db94f40 to a54ae8c Compare July 9, 2026 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants