Skip to content

feat(client)!: tool_calls mirrors the engine; attempts move to tool_call_attempts #117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mikasenghaas wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat(client)!: tool_calls mirrors the engine; attempts move to tool_call_attempts #117

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 14 additions & 9 deletions renderers/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -362,19 +362,23 @@ def _prepare():
routed_experts = choice.get("routed_experts")
kept_tokens = choice.get("kept_tokens")

# An engine's chat-completions endpoint only ever emits calls it accepted:
# vLLM's glm45/glm47 parsers run with ``validate_tool_names=True`` and drop
# the rest. ``tool_calls`` mirrors that — the calls a caller should execute —
# so an agent loop behaves the same whether it generated through this client
# or through the engine. Every attempt, accepted or not, stays available on
# ``tool_call_attempts`` for the uses engines can't serve: schema-adherence
# rubrics and selective token masking over non-OK spans.
#
# /inference/v1/generate returns finish_reason in {"stop","length",...} —
# never "tool_calls" (a chat-completions concept). Promote stop→tool_calls
# when we extracted at least one well-formed tool call client-side, so
# OpenAI-compatible agent loops continue past the tool turn instead of
# treating the response as final. Malformed attempts (INVALID_JSON,
# UNCLOSED_BLOCK, ...) don't qualify — those still surface on
# ``parsed.tool_calls`` so verifiers can inspect them, but they don't
# trigger the tool-loop continuation.
# when we extracted at least one executable call, so OpenAI-compatible agent
# loops continue past the tool turn instead of treating the response as final.
finish_reason = choice.get("finish_reason")
ok_tool_calls = [
emitted_tool_calls = [
tc for tc in parsed.tool_calls if tc.status == ToolCallParseStatus.OK
]
if ok_tool_calls and finish_reason == "stop":
if emitted_tool_calls and finish_reason == "stop":
finish_reason = "tool_calls"

return {
Expand All @@ -384,7 +388,8 @@ def _prepare():
"completion_logprobs": completion_logprobs,
"content": parsed.content,
"reasoning_content": parsed.reasoning_content,
"tool_calls": parsed.tool_calls,
"tool_calls": emitted_tool_calls,
"tool_call_attempts": parsed.tool_calls,

@cursor cursor Bot Aug 6, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Existing test asserts old tool_calls contract

Medium Severity

tool_calls now only carries OK attempts, but test_generate_does_not_promote_finish_reason_for_malformed_tool_calls still asserts a single INVALID_JSON entry under tool_calls. With the malformed-only renderer that list is empty now, so the suite fails until the assertion moves to tool_call_attempts.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit d4c2381. Configure here.

"finish_reason": finish_reason,
"routed_experts": routed_experts,
"kept_tokens": kept_tokens,
Expand Down
Loading