fix(streaming): wrap midstream transport errors in APITimeoutError / APIConnectionError (#3811) - #3813
Closed
katariyaVivek wants to merge 1 commit into
Closed
Conversation
…APIConnectionError (openai#3811)
marcuswood-oai
added a commit
that referenced
this pull request
Sep 10, 2026
## Summary A timeout or broken connection after streaming starts currently escapes as a raw HTTPX exception, bypassing `except openai.APIError`. Wrap request failures at the event-read boundary, preserving the original exception as `__cause__` and closing the response. Keep parsing and callback errors unchanged. Preserve existing Assistants event-handler and raw byte-stream behavior. Partially consumed streams are not retried. Fixes #3811. Supersedes the overlapping fixes in #3813, #3814, and #3818. ## Release note Release as a **minor version**. `Stream` and `AsyncStream` now raise `openai.APITimeoutError` for read timeouts and `openai.APIConnectionError` for other HTTPX request failures, including decoding errors. Applications catching raw `httpx` or `httpx2` exceptions during event streaming should catch these SDK exceptions instead. The underlying exception remains available through `__cause__`. ## Validation - Merged current main and resolved the test-import conflict. - Six after-first-chunk regression cases fail on main and pass with this change. - 77 focused tests pass on each of Pydantic v1 and v2, including sync/async HTTPX2 and legacy HTTPX, response cleanup, no retries, and callback compatibility. - Broad offline suite: 3,658 passed, 132 skipped; large-payload contract: 1 passed, run sequentially. - Ruff, Mypy, and focused Pyright pass. - Custom-code budget passes: 6,827 / 10,000 lines. - Full diff security review found no unrelated changes or dependency, credential, or workflow modifications. API/mock-server and network-dependent suites were excluded from the broad local run. The separate optional legacy aiohttp adapter test could not run because `httpx_aiohttp` is not installed. --------- Co-authored-by: Marcus Wood <marcuswood@openai.com>
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When consuming a streaming response (e.g.
client.chat.completions.create(..., stream=True)), transport errors that occur mid-stream (such as a read timeout or dropped socket connection) escaped as rawhttpx.ReadTimeout,httpx2.ReadTimeout, orhttpx2.RemoteProtocolError.Because these exceptions are not subclasses of
openai.APIError(they do not inherit fromAPIConnectionErrororAPITimeoutError), standard exception-handling blocks around API calls:fail to catch the most common streaming failures, breaking client resilience and leaving mid-stream errors inconsistent with the initial request-send path.
Root Cause
Stream.__stream__()andAsyncStream.__stream__()insrc/openai/_streaming.pyiterated over SSE events without exception-wrapping blocks. Unlike_base_client.py(which mapstimeout_exceptions()toAPITimeoutErrorand transport exceptions toAPIConnectionError), the streaming iterator let rawhttpx2/ legacyhttpxexceptions bubble out directly to the consumer.Solution
src/openai/_streaming.py, wrapped the iteration loop inStream.__stream__()andAsyncStream.__stream__():except timeout_exceptions() as err:->raise APITimeoutError(request=request) from errexcept _transport_exceptions() as err:->raise APIConnectionError(request=request) from err_transport_exceptions()covershttpx2.TransportErrorandhttpx.TransportError(when legacy httpx is loaded), matching thetimeout_exceptions()pattern.response.close()/await response.aclose()is safely retained in thefinally:block.src/openai/lib/streaming/_assistants.py, handledAPITimeoutErrorandAPIConnectionErrorinAssistantEventHandlerandAsyncAssistantEventHandler, unwrappingexc.__cause__so that existingon_timeout()andon_exception()callbacks receive the underlying transport exception family, preserving the contract tested bytest_assistant_stream_timeout_callbacks_preserve_httpx2_family.Testing
Added 4 new test cases to
tests/test_httpx2.py:test_chat_stream_midstream_timeout_wrapped(syncReadTimeout->APITimeoutError)test_chat_stream_midstream_connection_error_wrapped(asyncRemoteProtocolError->APIConnectionError)test_chat_stream_midstream_connection_error_wrapped_sync(syncRemoteProtocolError->APIConnectionError)test_chat_stream_midstream_timeout_wrapped_async(asyncReadTimeout->APITimeoutError)Verified:
httpx2.ReadTimeoutandhttpx2.RemoteProtocolErrorescape raw).__cause__correctly chained.tests/test_httpx2.pyandtests/test_streaming.pypass cleanly (41 passed, 1 skipped).Verification
tests/test_httpx2.pytests/test_httpx2.pyandtests/test_streaming.py_constants.py,_httpx2.py, and_exceptions.pyruff checkpassed with 0 errorsruff format --checkpassed (3 files checked)Impact
Consumers iterating over streams can now reliably catch
openai.APIError,openai.APIConnectionError, andopenai.APITimeoutErrorfor all midstream transport failures, consistent with the rest of the SDK.Fixes #3811