Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
End-user friendly description of the problem this fixes or functionality that this introduces.
This is ongoing work in file descriptor leaks, related to multiple PRs:
The previous issue was that there was some LiteLLM implementations which created multiple httpx clients and never closed any of them (Claude), some which closed their connections (Proxy), and some which used a semi global shared httpx client which is never closed (OpenAI).
In the last incarnation, the OpenAI model caused issues, which should now be resolved using the proxy client. The proxy client will close connections when asked, but transparently reopen them if requested (Similar to the requests library)
Once our LiteLLM PR is merged, we will be able to revisit (and hopefully remove) this.
Give a summary of what the PR does, explaining any non-trivial design decisions.
Link of any specific issues this addresses.
To run this PR locally, use the following command: