Fix StructuredOutputValidationAdvisor dropping tool call responses#6588
Open
subhashpolisetti wants to merge 1 commit into
Open
Conversation
After a JSON validation failure, a retry round that responded with a tool call request skipped validation but left the failure flag set, so the retry loop discarded the response and re-invoked the model with the same augmented request until the retry budget was exhausted. On the first attempt the same response shape was returned correctly only because the flag still held its initial value. Break out of the retry loop when the model responds with a tool call, so the response is handed back to the outer tool-calling advisor for execution, consistent with the first-attempt behavior. Signed-off-by: subhash polisetti <subhashr161347@gmail.com>
db94f40 to
a54ae8c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
By default
StructuredOutputValidationAdvisorruns atLOWEST_PRECEDENCE - 2000,inside the
ToolCallingAdvisorloop (HIGHEST_PRECEDENCE + 300). When structuredoutput validation is combined with tool calling and the model answers a validation
retry with a tool call, that response is silently discarded. The advisor re-invokes
the model with the same augmented prompt until
maxRepeatAttemptsis exhausted,wasting the remaining retry budget on calls whose tool-call responses are thrown
away, and delaying the tool execution the model asked for.
Root cause
The retry loop exits only when validation succeeds or the attempt budget runs out.
Tool-call responses correctly skip the validation block (per the existing "We should
not validate tool call requests" comment), but nothing ended the loop on that branch,
so the failure flag stayed set and every tool-call round was treated as another
failed attempt. On the first attempt it worked only by accident, because
isValidationSuccessstill held its initialtrue.Fix
Break out of the retry loop when the model responds with a tool call, so the response
is handed back to the outer
ToolCallingAdvisorfor execution, consistent with thefirst-attempt behavior. Validation still runs on every content response, and the
retry and augmentation logic is unchanged.
Testing
whenToolCallResponseFollowsValidationFailureThenReturnedForToolExecutionfails onthe current implementation, where the chain is invoked a third time after the tool
call (
expected: 2 but was: 3), and passes with the fix.whenToolCallResponseOnFirstAttemptThenReturnedWithoutValidationpins thealready-correct first-attempt behavior.
Both tests are deterministic and provider-free, using
DefaultAroundAdvisorChainwitha terminal advisor (the same harness as the existing tests). Existing tests are
unaffected, and
./mvnw -pl spring-ai-client-chat clean testpasses (571 tests).