Skip to content

fix(streaming): wrap midstream transport errors in APITimeoutError / APIConnectionError (#3811) - #3813

Closed
katariyaVivek wants to merge 1 commit into
openai:mainfrom
katariyaVivek:fix/stream-transport-error-wrapping
Closed

fix(streaming): wrap midstream transport errors in APITimeoutError / APIConnectionError (#3811)#3813
katariyaVivek wants to merge 1 commit into
openai:mainfrom
katariyaVivek:fix/stream-transport-error-wrapping

Conversation

@katariyaVivek

Copy link
Copy Markdown
  • I understand that this repository is auto-generated and my pull request may not be merged

Problem

When consuming a streaming response (e.g. client.chat.completions.create(..., stream=True)), transport errors that occur mid-stream (such as a read timeout or dropped socket connection) escaped as raw httpx.ReadTimeout, httpx2.ReadTimeout, or httpx2.RemoteProtocolError.

Because these exceptions are not subclasses of openai.APIError (they do not inherit from APIConnectionError or APITimeoutError), standard exception-handling blocks around API calls:

try:
    for chunk in client.chat.completions.create(..., stream=True):
        ...
except openai.APIError as err:
    ...

fail to catch the most common streaming failures, breaking client resilience and leaving mid-stream errors inconsistent with the initial request-send path.

Root Cause

Stream.__stream__() and AsyncStream.__stream__() in src/openai/_streaming.py iterated over SSE events without exception-wrapping blocks. Unlike _base_client.py (which maps timeout_exceptions() to APITimeoutError and transport exceptions to APIConnectionError), the streaming iterator let raw httpx2 / legacy httpx exceptions bubble out directly to the consumer.

Solution

  1. In src/openai/_streaming.py, wrapped the iteration loop in Stream.__stream__() and AsyncStream.__stream__():
    • except timeout_exceptions() as err: -> raise APITimeoutError(request=request) from err
    • except _transport_exceptions() as err: -> raise APIConnectionError(request=request) from err
    • _transport_exceptions() covers httpx2.TransportError and httpx.TransportError (when legacy httpx is loaded), matching the timeout_exceptions() pattern.
    • response.close() / await response.aclose() is safely retained in the finally: block.
  2. In src/openai/lib/streaming/_assistants.py, handled APITimeoutError and APIConnectionError in AssistantEventHandler and AsyncAssistantEventHandler, unwrapping exc.__cause__ so that existing on_timeout() and on_exception() callbacks receive the underlying transport exception family, preserving the contract tested by test_assistant_stream_timeout_callbacks_preserve_httpx2_family.

Testing

Added 4 new test cases to tests/test_httpx2.py:

  • test_chat_stream_midstream_timeout_wrapped (sync ReadTimeout -> APITimeoutError)
  • test_chat_stream_midstream_connection_error_wrapped (async RemoteProtocolError -> APIConnectionError)
  • test_chat_stream_midstream_connection_error_wrapped_sync (sync RemoteProtocolError -> APIConnectionError)
  • test_chat_stream_midstream_timeout_wrapped_async (async ReadTimeout -> APITimeoutError)

Verified:

  • Fails pre-fix (httpx2.ReadTimeout and httpx2.RemoteProtocolError escape raw).
  • Passes post-fix with __cause__ correctly chained.
  • tests/test_httpx2.py and tests/test_streaming.py pass cleanly (41 passed, 1 skipped).

Verification

  • Targeted tests: 4 passed in tests/test_httpx2.py
  • Integration tests: 41 passed across tests/test_httpx2.py and tests/test_streaming.py
  • Type checking: Verified against _constants.py, _httpx2.py, and _exceptions.py
  • Lint: ruff check passed with 0 errors
  • Formatter: ruff format --check passed (3 files checked)

Impact

Consumers iterating over streams can now reliably catch openai.APIError, openai.APIConnectionError, and openai.APITimeoutError for all midstream transport failures, consistent with the rest of the SDK.

Fixes #3811

@katariyaVivek
katariyaVivek requested a review from a team as a code owner September 7, 2026 14:22
marcuswood-oai added a commit that referenced this pull request Sep 10, 2026
## Summary

A timeout or broken connection after streaming starts currently escapes
as a raw HTTPX exception, bypassing `except openai.APIError`. Wrap
request failures at the event-read boundary, preserving the original
exception as `__cause__` and closing the response.

Keep parsing and callback errors unchanged. Preserve existing Assistants
event-handler and raw byte-stream behavior. Partially consumed streams
are not retried.

Fixes #3811. Supersedes the overlapping fixes in #3813, #3814, and
#3818.

## Release note

Release as a **minor version**. `Stream` and `AsyncStream` now raise
`openai.APITimeoutError` for read timeouts and
`openai.APIConnectionError` for other HTTPX request failures, including
decoding errors. Applications catching raw `httpx` or `httpx2`
exceptions during event streaming should catch these SDK exceptions
instead. The underlying exception remains available through `__cause__`.

## Validation

- Merged current main and resolved the test-import conflict.
- Six after-first-chunk regression cases fail on main and pass with this
change.
- 77 focused tests pass on each of Pydantic v1 and v2, including
sync/async HTTPX2 and legacy HTTPX, response cleanup, no retries, and
callback compatibility.
- Broad offline suite: 3,658 passed, 132 skipped; large-payload
contract: 1 passed, run sequentially.
- Ruff, Mypy, and focused Pyright pass.
- Custom-code budget passes: 6,827 / 10,000 lines.
- Full diff security review found no unrelated changes or dependency,
credential, or workflow modifications.

API/mock-server and network-dependent suites were excluded from the
broad local run. The separate optional legacy aiohttp adapter test could
not run because `httpx_aiohttp` is not installed.

---------

Co-authored-by: Marcus Wood <marcuswood@openai.com>
@marcuswood-oai

Copy link
Copy Markdown
Contributor

Thanks for working on this! We merged #3827 to address #3811, so I’m closing this as a duplicate. Appreciate the contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Transport errors while consuming a stream escape as raw httpx exceptions instead of APITimeoutError / APIConnectionError

2 participants