[Bug]: Chat JSON-to-SSE fallback drops tool calls/reasoning and overwrites finish_reason with stop

[turin-dev]

Resolved in dev

#3803 (ac4a7659fd) preserves tools, reasoning, usage and finish semantics in JSON-to-SSE completion delivery.

Verified against dev 5759d9ea2f1e7281cdc01eb9628f2e0a123fb59c. Original report by @turin-dev. The attribution record was added in #3811.

Client or integration

Direct HTTP/API client

Area

Streaming

Summary

When the Chat endpoint's internal Responses replay returns JSON instead of SSE, stream: true enters a lossy JSON-to-SSE fallback. The fallback serializes only message.content and hard-codes finish_reason: "stop", even though responsesJsonToChatCompletion has already produced tool calls, reasoning content, and the correct finish reason.

The same upstream JSON produces:

• stream: false: the expected message.tool_calls and finish_reason: "tool_calls".
• stream: true: no tool-call delta at all; just the assistant role and a successful-looking stop followed by [DONE].

A reasoning-plus-text response loses its reasoning_content while retaining normal text. A response incomplete due to max_output_tokens changes from length to stop.

Impact: tool-capable clients cannot execute the requested tool, and clients cannot detect token-limit truncation from the terminal chunk. This is specific to the JSON response fallback, not a claim about the usual SSE-to-SSE or native Chat passthrough paths.

Expected: the synthesized stream preserves the semantic fields and terminal reason of the already-converted Chat completion: indexed delta.tool_calls, delta.reasoning_content, and choices[0].finish_reason.

Reproduction

Executed the unmodified handleChatCompletions and translation modules from commit a349b521bb02f76c7b8820c13633a8076dcec378 with a mocked handleResponses boundary that returns application/json. Routing and unrelated runtime integrations are stubbed so the test needs no provider, credentials, or running proxy.

Save the following as repro-json-chat.ts in the repository root, then run it in a separate process with bun run repro-json-chat.ts (the mocks are process-global):

import { mock } from "bun:test";
let upstreamBody: Record<string, unknown>;
mock.module("./src/adapters/openai-responses", () => ({ FORWARD_HEADERS: [] }));
mock.module("./src/lib/retry-after", () => ({ resolveClientRetryAfter: () => undefined }));
mock.module("./src/lib/token-estimate", () => ({ estimateTokens: () => 1 }));
mock.module("./src/router", () => ({
  UnknownRoutingPolicyError: class extends Error {},
  NoEligiblePolicyCandidateError: class extends Error {},
  routeModel: () => ({
    providerName: "fixture", modelId: "fixture", codexAccountMode: "direct",
    provider: { adapter: "openai-responses" },
  }),
}));
mock.module("./src/routing/request-evidence", () => ({ evidenceFromBody: () => ({}) }));
mock.module("./src/server/adapter-resolve", () => ({ resolveWireProtocolOverride: (_name, _model, provider) => provider }));
mock.module("./src/server/request-decompress", () => ({ readJsonRequestBody: (req) => req.json() }));
mock.module("./src/server/request-log", () => ({
  addFinalRequestLog() {}, httpStatusForRequestLogTerminal: () => 200, recordFirstOutput() {},
}));
mock.module("./src/server/relay", () => ({ responseWithDeferredRequestLog: (r) => r }));
mock.module("./src/server/responses", () => ({ handleResponses: async () => Response.json(upstreamBody) }));
mock.module("./src/codex/native-main-admission", () => ({ tryClaimNativeMainProfileForTurn: () => false }));
mock.module("./src/server/chat-native", () => ({ isNativeChatRouteEligible: () => false, handleNativeChatCompletions() { throw Error("unexpected native path"); } }));
mock.module("./src/server/effort-row", () => ({ parseRequestEffortRowId: () => null }));
mock.module("./src/server/fast-row", () => ({ parseSyntheticRowId: () => ({}) }));
mock.module("./src/providers/openai-tiers", () => ({ isCanonicalOpenAiForwardProvider: () => false }));
mock.module("./src/codex/loopback-target", () => ({ CODEX_RESERVE_HELPER_UNSUPPORTED_MESSAGE: "", isCodexReserveHelperUnsupported: () => false }));
const { handleChatCompletions } = await import("./src/server/chat-completions");
const fixtures = {
  tool: { status: "completed", output: [{ type: "function_call", call_id: "call_fixture", name: "lookup", arguments: '{"q":"hello"}' }] },
  reasoning: { status: "completed", output: [{ type: "reasoning", summary: [{ type: "summary_text", text: "Fixture reasoning." }] }, { type: "message", role: "assistant", content: [{ type: "output_text", text: "Answer." }] }] },
  truncated: { status: "incomplete", incomplete_details: { reason: "max_output_tokens" }, output: [{ type: "message", role: "assistant", content: [{ type: "output_text", text: "Partial answer." }] }] },
};
for (const [name, fixture] of Object.entries(fixtures)) {
  upstreamBody = { id: "resp_fixture", ...fixture, usage: { input_tokens: 1, output_tokens: 2 } };
  for (const stream of [false, true]) {
    const req = new Request("http://localhost/v1/chat/completions", {
      method: "POST", headers: { "content-type": "application/json" },
      body: JSON.stringify({ model: "fixture", messages: [{ role: "user", content: "Hello" }], stream }),
    });
    const res = await handleChatCompletions(req, {} as any, {} as any);
    const text = await res.text();
    if (!stream) {
      console.log(name, "JSON", res.status, JSON.stringify(JSON.parse(text).choices[0]));
    } else {
      const frames = text.split("\n\n").filter(s => s.startsWith("data: {")).map(s => JSON.parse(s.slice(6)));
      console.log(name, "SSE", res.status, JSON.stringify(frames.map(f => f.choices[0])));
    }
  }
}

This exercises six handler calls: three identical upstream fixtures, each with the client's stream flag false and true. The real request handler returns HTTP 200 for all six. It is a handler-level reproduction with injected upstream responses, not a live-provider/full-server integration test.

Version

Upstream dev commit a349b521bb02f76c7b8820c13633a8076dcec378, checked 2026-09-06.

Operating system

Ubuntu 24.04.3 LTS, Linux x64; Bun 1.4.2.

Provider and model

Synthetic Responses upstream; model identifier fixture. Relevant when the internal Responses replay returns JSON to a streaming Chat request.

Logs or error output

Normalized observed differences (generated IDs omitted):

Fixture	Client stream=false	Client stream=true
Completed function call	message.tool_calls[0].function.name = "lookup"; finish tool_calls	No delta.tool_calls; finish stop
Reasoning plus text	reasoning_content = "Fixture reasoning." and content = "Answer."	Text remains; no reasoning delta
Token-limited partial answer	content = "Partial answer."; finish length	Same partial text; finish stop

For the tool fixture, the entire streamed choice sequence is:

[
  {"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null},
  {"index":0,"delta":{},"finish_reason":"stop"}
]

It ends with [DONE], without ever transmitting the call ID, function name, or arguments.

Screenshots and supporting files

Root cause: src/server/chat-completions.ts, JSON-to-SSE fallback.

• The completed Chat object already exists at line 450.
• Lines 462–465 extract only message.content.
• Lines 466–470 emit role/text chunks only.
• Line 471 replaces the actual finish reason with the constant "stop".

Suggested fix: serialize the full supported Chat completion message into indexed deltas and reuse the converted choice's finish reason. Keep usage and one terminal [DONE], with the existing stream-budget/lifecycle requirements. Add focused handler tests for tool-only JSON, multiple tool calls, reasoning plus text, and max-output-token truncation.

Distinct from #3767: that report concerns structured refusal content lost inside src/chat/outbound.ts. Here the JSON-to-Chat conversion successfully preserves the tool/reasoning fields, and the subsequent SSE synthesis in the HTTP handler discards them.

Duplicate checks included issue searches for JSON fallback/upstream and finish_reason, plus PR searches for JSON fallback, JSON upstream + tool, and synthesized Chat streams. No matching fix/report was found; the defect remains in the pinned current-dev source.

Redacted configuration

No real configuration is needed. Routing is a synthetic direct Responses fixture in the harness. All identifiers, text, function names, arguments, and usage values are synthetic.

Checks

• I searched existing issues and documentation.
• I removed secrets, tokens, account details, request credentials, and personal data.

[lidge-jun]

리뷰 · 우선순위 72 / 80

이 이슈는 /v1/chat/completions에서 클라이언트가 stream: true를 보냈는데, 내부 Responses 재생이 SSE가 아니라 JSON으로 돌아올 때 타는 JSON→SSE 합성 폴백이 이미 만든 Chat completion의 의미를 버리는 버그입니다. 같은 업스트림 JSON이 stream: false면 message.tool_calls와 finish_reason: "tool_calls"가 그대로 나오고, stream: true면 tool-call delta가 한 번도 없이 assistant 역할과 finish_reason: "stop"만 나갑니다. reasoning이 섞인 답은 reasoning_content가 사라지고 본문만 남고, max_output_tokens로 끊긴 답은 length가 stop으로 바뀝니다. 도구를 쓰는 클라이언트는 “성공한 빈 stop”처럼 보여서 호출을 실행하지 못하고, 토큰 한도 잘림도 마지막 청크로 감지하지 못합니다. 리포터가 말한 대로 평소 SSE→SSE 번역이나 네이티브 Chat 패스스루 문제가 아니라, HTTP 핸들러의 JSON 폴백 합성만의 구멍입니다.

지금 dev HEAD는 bd1cda99c(package 2.44.0, #3768 수동·자동 credential 선택 공유까지 들어간 release-244). 리포터 기준 커밋 a349b521b는 현재 HEAD의 조상이고, HEAD의 src/server/chat-completions.ts L450–L480 근처도 같은 코드입니다. 흐름은 이렇습니다. handleChatCompletions가 내부적으로 Responses를 돌린 뒤 JSON을 받으면 responsesJsonToChatCompletion(정의는 src/chat/outbound.ts L563 근처)으로 이미 완전한 Chat completion을 만듭니다. 그 함수는 tool call을 message.tool_calls로 모으고, reasoning summary/text를 reasoning_content에 넣고, finish reason을 tool_calls / length / stop으로 고릅니다(대략 L605–L624). stream: false는 그 객체를 그대로 JSON으로 돌려주니 맞습니다. 문제는 바로 아래 스트리밍 합성입니다. L462–L465에서 message.content 문자열만 꺼내고, L466–L470은 role/text 청크만 만들고, L471은 finish_reason을 상수 "stop"으로 덮어씁니다. 그래서 변환기가 이미 살려 둔 tool/reasoning/정확한 종료 이유가 HTTP 응답에서 증발합니다.

평소 스트리밍 경로인 src/chat/outbound.ts의 responsesSseToChatCompletionsSse는 이미 delta.tool_calls, delta.reasoning_content, 그리고 올바른 finish_reason을 냅니다(대략 L245–L307, L448). 즉 “Chat로 번역하는 능력”은 레포에 있고, JSON 폴백만 그 능력을 다시 쓰지 않는 상태입니다. #3767과도 겹치지 않습니다. #3767은 outbound.ts 번역 단계에서 refusal 텍스트가 빠지는 문제이고, 여기서는 JSON→Chat 변환은 tool/reasoning을 이미 보존한 뒤 HTTP 핸들러 합성이 버립니다. types/config 분할 캠페인과도 무관해서 close-don't-rebase 대상이 아닙니다. 재현 스크립트도 handleChatCompletions 경계에 handleResponses만 JSON으로 mock한 핸들러 단위라서, 라이브 프로바이더 없이도 고치기 좋은 형태입니다.

라인 450 - responsesJsonToChatCompletion으로 이미 tool_calls·reasoning·finish_reason이 있는 completion이 생기는데, 바로 아래 합성은 이 객체를 “텍스트만 있는 가짜 스트림”으로 깎아 냄
라인 465 - message.content 문자열만 추출해서 tool_calls / reasoning_content / content가 null인 tool-only 케이스를 전부 놓침
라인 466-470 - role·텍스트 delta만 내보내서 indexed delta.tool_calls와 delta.reasoning_content 프레임이 한 번도 안 나감
라인 471 - 변환기가 고른 finish reason을 무시하고 상수 "stop"으로 덮어써서 tool_calls·length가 클라이언트로 전달되지 않음
경로 src/chat/outbound.ts responsesJsonToChatCompletion - 비스트리밍·변환 쪽은 이미 올바른데, 핸들러 폴백만 그걸 재사용하지 않아 stream true/false 의미가 갈라짐
경로 src/chat/outbound.ts responsesSseToChatCompletionsSse - SSE 경로에는 tool/reasoning emitter가 이미 있으므로, JSON 폴백은 이 시맨틱을 복제하거나 completion→chunk 직렬화 헬퍼를 공유하는 쪽이 안전함

메인테이너의 판단이 필요한 지점

• JSON 폴백을 outbound.ts의 기존 SSE emitter 시맨틱에 맞춰 공용 헬퍼로 뺄지, chat-completions.ts 안에서만 completion.choices[0]을 chunk로 펼칠지
• reasoning_content를 폴백 스트림에 넣을지(비스트리밍·SSE 경로와 맞추려면 넣는 편이 일관됨). 일부 클라이언트가 모르는 필드를 무시하는지만 확인
• 여러 tool call·tool-only(본문 null)·reasoning+text·max_output_tokens incomplete를 핸들러 단위 테스트로 고정할지(리포터 제안 그대로 채택해도 됨)
• #3767 refusal 수정과 같은 PR에 묶을지, JSON 폴백만 따로 짧게 고칠지(범위가 달라서 분리가 더 읽기 쉬움)

너의 추천
수락하고 고치세요. src/server/chat-completions.ts JSON→SSE 폴백에서 이미 만든 completion의 message.tool_calls·reasoning_content·choices[0].finish_reason을 indexed delta로 직렬화하고, 하드코딩 "stop"을 제거하세요. usage와 마지막 [DONE]·기존 stream-budget/lifecycle은 유지한 채, tool-only·다중 tool call·reasoning+text·length truncation 핸들러 테스트를 추가하면 release-244에도 안전하게 들어갈 수 있습니다. #3767과는 별 이슈로 두고, 이 이슈만으로 PR을 받는 걸 추천합니다.

이 댓글은 grok-bot이 작성했습니다

[Ingwannu]

Confirmed on current dev (bd1cda99c) with a local synthetic JSON upstream and the actual handleChatCompletions path. All four new handler regressions fail before the fix: one tool call, two indexed tool calls, reasoning plus answer text, and an incomplete response's length finish reason. No provider account or production configuration is involved.

I am preparing a focused patch to serialize the already-converted completion's supported message fields into Chat deltas and retain its finish reason, usage, and one [DONE]. The native Chat route and actual SSE-upstream translator will remain unchanged. This is separate from #3767's earlier refusal-conversion loss.

[Ingwannu]

Implementation is in Draft PR #3779 (head 7362d77). The four previously failing handler regressions now pass; the expanded focused run is 24 pass / 0 fail, including ordinary text, empty output, cancellation and layout guards. TypeScript, privacy scan, and the 425-page docs build also pass. Full-suite and platform CI validation remain pending, so this is not merged or released and the issue stays open. The change is limited to the JSON-to-SSE fallback; #3767 remains separate.

[lidge-jun]

Landed via #3803 at ac4a7659fd833ad89c525497abd7847640c30b53 (Chat JSON-to-SSE completion semantics on dev).

[lidge-jun]

Resolved on dev by #3803, with the dependent refusal coverage in #3805. The actual Chat handler now retains indexed tools, reasoning, usage and finish semantics in buffered JSON-to-SSE fallback, with bounded admission and typed failure handling. Final combined cross-platform verification: https://github.com/lidge-jun/opencodex/actions/runs/34058496989 .