Skip to content

responses: bridged tool identity is lost mid-stream — frame boundaries, argument fragments and history replay all break the call #6218

Closed

Description

@hrygo

Is it a request payload issue?
[ ] Yes, this is a request payload issue. I am using a client/cURL to send a request payload, but I received an unexpected error.
[x] No, it's another issue.

If it's a request payload issue, you MUST know
N/A — the request is valid and the first chunks stream correctly. The loss happens mid-stream, so request-log: true plus the client's own transcript is what shows it: the client receives a tool call whose name it never declared, or a stream that ends without the tool result it is waiting for.

Describe the bug

Summary

A bridged tool call has to survive being taken apart and put back together: SSE frames arrive in arbitrary chunk boundaries, a tool name is flattened and possibly capped for the upstream, arguments arrive as JSON string fragments across many deltas, and the whole conversation is replayed on the next turn. Every one of those steps can lose the link between the name on the wire and the tool the client declared. The failure is silent — the client sees a tool call it does not recognise, or a stream that simply stops.

Observed behavior

  • Mid-stream identity loss. The upstream returns a tool call under a flattened or capped name. If the mapping back to the canonical identity is done at request time instead of response time, the client receives a name it never declared and the conversation diverges silently.
  • Argument fragmentation. Tool arguments arrive as partial JSON string fragments. If a frame boundary lands mid-string, the accumulated arguments are corrupted or dropped, and the tool call is emitted with unparseable arguments.
  • Terminal-payload completion. Some upstreams deliver the complete arguments only in the terminal event for an item. A bridge that only accumulates deltas emits a tool call with empty arguments, or buffers the item forever and the stream never terminates.
  • Cross-turn replay drift. A tool that was discovered in turn 1 is called in turn 2. If the replayed history is not realigned with the alias currently active, the model sees a call to a tool that is not declared, and upstreams reject it or the model hallucinates a new one.
  • Unbounded per-item state. A long stream with many open items accumulates tracking state that is never released.

CLI Type
All bridged routes; most visible on streaming (stream: true) Responses requests.

Model Name
All.

LLM Client
Codex-style clients, streaming.

Request Information

The frames that must be handled. Note that a frame boundary can fall anywhere, including inside the arguments string:

event: response.output_item.done
data: {"type":"response.function_call_arguments.delta","item_id":"fc_1","delta":"{\"path\":"}

event: response.function_call_arguments.done
data: {"type":"response.function_call_arguments.done","item_id":"fc_1","arguments":"{\"path\":\"config.yaml\"}"}

event: response.output_item.added
data: {"type":"response.output_item.added","item_id":"fc_1","item":{"type":"function_call","name":"<flattened alias>","call_id":"call_1","arguments":""}}

Expected behavior

  • Frame boundaries are handled independently of content: a blank line delimits a frame, and CRLF counts as one line ending even when its two bytes arrive in separate chunks.
  • Both {"type": ...} event objects and {"event": ..., "data": ...} envelopes are supported. The SSE event name and the JSON type travel together; a mismatched envelope is rejected rather than guessed.
  • Tracked state is keyed by the attempt-local item_id. output_index and call_id only verify consistency — they never select the entry, because both are reused across rounds.
  • A tool call is emitted only when its arguments are complete. Items whose arguments are complete in the terminal payload are flushed at Finish; an incomplete tail is rejected rather than emitted.
  • Buffered state is a deliberate Feed returning zero events, and is distinguishable from pass-through: a nil attempt passes frames through, an active attempt buffers.
  • Replayed history is realigned with the currently active alias, touching only identities explicitly deferred or discovered in this request.
  • Every reservation is released when the stream ends, and Close is idempotent.

Screenshots
N/A.

OS Type

  • OS: macOS
  • Version: 26.x / arm64 (not platform specific)

Additional context

Scope in this issue

  • The stream adapter for one attempt: feed one decoded SSE event or plain JSON frame, adapt, and release every reservation on close.
  • Argument fragment accumulation with completion detection, terminal-payload flush, and incomplete-tail rejection.
  • Per-item tracking state with a fixed accounted overhead per entry, charged against the attempt's byte budget.
  • History-replay realignment and identity restoration on the response path, kept deliberately narrow: eager tools, unknown calls, and ambiguous names are left untouched.
  • The response-side guard used by the outbound check: it asserts the client-visible shape of a restored search item.

Out of scope

Request-side rewriting and discovery mechanics, which are covered by the search-bridge issue.

Invariants

  • No tool is ever emitted with incomplete arguments.
  • A frame boundary never changes the meaning of a frame.
  • Ambiguity never resolves.

Related issues

Depends on the contract/identity representation; read with the search-bridge and custom-tool-bridge issues, whose response paths this completes.

Activity

  1. hrygo commented on Sep 30, 2026

    @hrygo
    Author

    Closing this — we continue the work in our own fork.

    Since 2026-09-30 we maintain https://github.com/hrygo/CLIProxyAPI as our source of truth: our own trunk, our own branches, our own issue tracking and releases. We still follow this repository and pick up its releases, but we no longer file or carry our own issues and PRs here, so this one is closed and continued in the fork.

    If this is useful for upstream itself, it is still welcome to report here. For our own development and bug reports, please use the fork.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    pendingWaiting for research

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions