Skip to content

fix(translation): preserve reasoning order in mixed stream chunks - #387

Open
Blakeolson21 wants to merge 3 commits into
NVIDIA-NeMo:mainfrom
Blakeolson21:fix/stream-reasoning-order
Open

fix(translation): preserve reasoning order in mixed stream chunks#387
Blakeolson21 wants to merge 3 commits into
NVIDIA-NeMo:mainfrom
Blakeolson21:fix/stream-reasoning-order

Conversation

@Blakeolson21

@Blakeolson21 Blakeolson21 commented Aug 12, 2026

Copy link
Copy Markdown

What

Preserve reasoning-before-content order when an OpenAI Chat stream chunk contains both
reasoning_content and visible content.

Add a focused OpenAI Chat to Anthropic Messages stream regression test using the one-chunk
reproducer from Issue 242. The test asserts the complete thinking block, thinking close, and text
block event sequence.

Why

Reasoning models can return reasoning and final content in the same stream delta. Emitting visible
content first makes the Anthropic translation close the text block and open a later thinking block,
which can interleave reasoning with the final answer. We hit this while streaming reasoning models
through the translation layer.

Closes #242

How tested

  • uv run ruff check . clean
  • uv run mypy switchyard clean
  • uv run pytest tests/ green (134 passed, 2 deselected with integration tests excluded)
  • Manual smoke (not run; the issue's exact chunk is covered by the cross-format regression test)
  • cargo fmt --all --check clean
  • cargo clippy --workspace --all-targets -- -D warnings clean
  • cargo test --workspace green
  • Focused regression test demonstrated red before the decoder change and green after it

Checklist

  • One class per file; filename = snake_case of the primary class. No new classes or files.
  • New public symbols exported from switchyard/__init__.py.__all__ if intended for downstream use. No new public symbols.
  • Unit tests added for new components / bug fixes.
  • README / --help updated if customer-facing surface changed. No customer-facing API surface changed.
  • Commits signed off (Signed-off-by: Your Name <email>) per the DCO.

Notes for reviewers

The production diff only moves existing content delta emission below the existing reasoning-field
loop. Parsing, field precedence, tool calls, finish reasons, and error behavior are unchanged.

The regression test asserts the exact Anthropic event order from the reported reproducer: thinking
block start, reasoning delta, signature delta, thinking block stop, text block start, and text delta.
Chunks containing only reasoning or only content retain their existing output.

Summary by CodeRabbit

  • Bug Fixes

    • Streaming responses now consistently deliver reasoning content before text when both arrive together.
    • Text extraction behavior remains unchanged.
  • Tests

    • Added coverage to verify the correct ordering of reasoning and text content in translated streaming responses.

Signed-off-by: Blake <Blakeolson5@gmail.com>
@Blakeolson21
Blakeolson21 requested a review from a team as a code owner August 12, 2026 18:49
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8b79a950-8cef-4051-8ce9-a3a34fa9ce3a

📥 Commits

Reviewing files that changed from the base of the PR and between 48b3b71 and bcc26f0.

📒 Files selected for processing (2)
  • crates/switchyard-translation/src/codecs/openai_chat/stream.rs
  • crates/switchyard-translation/tests/stream_translation.rs

Walkthrough

The OpenAI Chat stream decoder now emits reasoning before text when both fields occur in one delta. A regression test verifies the translated Anthropic events: thinking block, reasoning deltas, block closure, then text block and delta.

Changes

Stream event ordering

Layer / File(s) Summary
Mixed-delta ordering and regression coverage
crates/switchyard-translation/src/codecs/openai_chat/stream.rs, crates/switchyard-translation/tests/stream_translation.rs
The decoder emits reasoning content before text content within one delta. The regression test verifies the corresponding Anthropic event order.

Estimated code review effort: 2 (Simple) | ~10 minutes

Poem

I’m a rabbit with a stream-wise cheer,
Reasoning hops before text appears.
Thinking starts, then thinking ends,
Clear words follow as trusted friends.
One neat order—carrots near! 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes preserving reasoning order in mixed streaming chunks, which is the primary change.
Linked Issues check ✅ Passed The code reorders reasoning before text and adds the required regression test for mixed OpenAI stream chunks.
Out of Scope Changes check ✅ Passed All changes directly support issue #242 by fixing stream ordering and testing the expected Anthropic event sequence.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

Comment @coderabbitai help to get the list of available commands.

Warning

⚠️ This pull request shows signs of AI-generated slop (redundant_comments). It has been flagged by CodeRabbit slop detection and should be reviewed carefully.

Signed-off-by: Blake <Blakeolson5@gmail.com>
Signed-off-by: Blake <Blakeolson5@gmail.com>
@Blakeolson21

Copy link
Copy Markdown
Author

Thanks for the review. 25da6fe removes the redundant test doc comment, and f9cb482 merges current main without conflicts. The full local Rust and Python gates are green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] Mixed reasoning/content stream chunks are reordered for Anthropic clients

1 participant