Skip to content

review: post-v0.2.1 double check — tool_choice across wires, stop_reason on tool_use, docs - #94

Merged
CMGS merged 11 commits into
mainfrom
review/post-v0.2.1-double-check
Sep 18, 2026
Merged

CMGS merged 11 commits into
mainfrom
review/post-v0.2.1-double-check

Conversation

@CMGS

@CMGS CMGS commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

What

Static double check of everything merged since PR 90 / v0.2.1 (PRs 91, 92, 93), plus the live checks the delta called for. Eleven commits: five fixes, two feature commits, three docs, one rustfmt.

  • engines: an Anthropic-shaped tool_choice ({type: auto|any|none|tool}) reached the OpenAI chat wire verbatim and the vendor answered 400 "Missing required parameter: 'tool_choice.function'"; a chat-shaped {type: function, function: {name}} reached the Responses wire the same way ("Missing required parameter: 'tool_choice.name'"). Both convert now; the reverse direction (normalize_tool_choice_anthropic) already did.
  • engines: the parallel tool policy travels with the choice in both directions — disable_parallel_tool_use becomes parallel_tool_calls on the OpenAI chat and Responses wires, and an OpenAI client's parallel_tool_calls becomes tool_choice.disable_parallel_tool_use on the Messages wires (anthropic-messages, aws-anthropic) instead of riding the raw extras into a 400 "parallel_tool_calls: Extra inputs are not permitted". Converse has no such flag in toolConfig, so there the field stays in additionalModelRequestFields for the model to accept or reject: the OpenAI family accepts it, Anthropic models answer 400. The normalizer stays a pure shape conversion; the policy comes from parallel_tool_calls(&choice).
  • views: /v1/messages said end_turn next to a tool_use block when OpenAI reported stop for a forced tool, so an Anthropic client would not run the tool. end_turn becomes tool_use when the content carries a tool_use block, on the buffered and the streamed path; max_tokens and the other reasons stay.
  • docs: the request body cap joins the resource-limit list in the security model; tools/tool_choice/parallel-policy conversion across wires and the derived stop_reason are documented in the API page.

Static review (delta c2786c6..a66250a)

Every changed production file read in full: config, protocol/openai, engines/openai_engine, engines/families, views. No finding in the delta itself. PR 92's tool_calls loosening (typed struct → Vec<Value>) was checked against every unchanged consumer of client-verbatim tool calls — token estimate, the two content walks, tool_calls_to_tool_use, the claude and openai engines — all match on the variant or filter before indexing, none can panic. Comment budget on the delta: one field doc, one doc rewritten, zero inline comments. This PR adds four one-line /// docs and no inline comments.

Live evidence (Linux testbed, real Gemini, OpenAI, Anthropic and Bedrock, 22/22 on this head's production code, gateway logs 0 WARN/ERROR)

PR check result
92 gemini-3.5-flash through the OpenAI-compatible endpoint: the tool call carries extra_content.google.thought_signature; echoing the turn verbatim is accepted; stripping the field is refused by Google (400) 3/3
92 same on the stream: the fragment carries the signature, the assembled turn is accepted 2/2
93 outbound DLP hit on model-written tool arguments (gpt-4o-mini): the synthesized replay is one frame with index 0 and 1, arguments redacted; /v1/messages renders one tool_use block per call 2/2
91 default cap: a 3 MiB body reaches the vendor, 33 MiB is 413; max_request_bytes: 4194304: 5 MiB is 413, 3 MiB passes 4/4
this PR /v1/messages tool_choice {type: auto} / {type: any} / {type: tool} over gpt-4o-mini → 200 with stop_reason: tool_use, buffered and streamed; chat {type: function, function: {name}} over gpt-4.1-mini on the Responses wire → 200 with one call 5/5
this PR chat parallel_tool_calls: false over claude-haiku-4-5 → 200 with a tool call; /v1/messages tool_choice with disable_parallel_tool_use over gpt-4o-mini → 200 with a tool_use block 2/2
this PR Bedrock us-east-1, chat parallel_tool_calls: false: aws-anthropic InvokeModel haiku-4-5 → 200 (converted); Converse gpt-oss-20b → 200 (native field); Converse haiku-4-5 → 424 carrying the vendor's 400 "parallel_tool_calls: Extra inputs are not permitted" (the documented reject case; the same request without the field → 200) 4/4

Before the fixes the five tool_choice cases answered 424 carrying the vendor 400s quoted above, the forced-tool reply said end_turn, and the Anthropic parallel_tool_calls case answered 424 with the vendor's "Extra inputs are not permitted".

Hot path

tool_choice normalization and the policy lookup run once per request and only when the field is present (one match; one small map for the tool case). The stop_reason derivation is one pass over the reply's content blocks on /v1/messages. The chat path is unchanged.

Gates

cargo fmt --check clean; cargo clippy --workspace --all-targets -- -D warnings clean; cargo test --workspace: 32 suites, 630 passed, 0 failed (engines re-run after the last two commits: 214 passed); Linux release build for the live run.

CMGS added 11 commits September 18, 2026 16:22
listen.max_request_bytes (PR 91) belongs next to max_reply_bytes and the x-gw-user cap in the security failure postures.
An Anthropic-shaped tool_choice ({type: auto|any|none|tool}) reached the
OpenAI chat wire verbatim and the vendor answered 400 "Missing required
parameter: 'tool_choice.function'"; a chat-shaped {type: function,
function: {name}} reached the Responses wire the same way ("Missing
required parameter: 'tool_choice.name'"). The reverse direction already
normalized (normalize_tool_choice_anthropic). Seen live on gpt-4o-mini
and gpt-4.1-mini through /v1/messages and /v1/chat/completions.
OpenAI reports finish_reason "stop" when tool_choice forces one function,
so the messages surface said end_turn next to a tool_use block and an
Anthropic client would not run the tool. end_turn becomes tool_use when
the content carries a tool_use block, on the buffered and the streamed
path; max_tokens and the other reasons stay. Seen live on gpt-4o-mini
with tool_choice {type: tool}.
normalize_tool_choice_openai took the request body to write
parallel_tool_calls, which forced a two-step insert at both call sites
and a scratch map in every test. The policy now comes from
parallel_tool_calls(&choice) and the caller inserts it; the normalizer
converts the shape and nothing else. Same wire output.
An OpenAI client's parallel_tool_calls rode the raw extras onto the
Messages body and Anthropic answered 400 "parallel_tool_calls: Extra
inputs are not permitted"; on Converse it landed in
additionalModelRequestFields. It now becomes
tool_choice.disable_parallel_tool_use (tool_choice defaults to auto,
type none is left alone), the mirror of the OpenAI-wire direction. Seen
live on claude-haiku-4-5 through /v1/chat/completions.
@CMGS
CMGS merged commit 554d041 into main Sep 18, 2026
2 checks passed
@CMGS
CMGS deleted the review/post-v0.2.1-double-check branch September 18, 2026 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant