httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)

[DHaru85]

using dumps_kwargs keyword arguments are no longer supported.

while deploying and testing qwen1.5-7b-chat through fastchat v0.2.36, i got an exception :

2024-08-16 03:11:13 | INFO | stdout | INFO:     172.22.0.1:48704 - "POST /v1/chat/completions HTTP/1.1" 200 OK
2024-08-16 03:11:13 | ERROR | stderr | /usr/local/lib/python3.10/site-packages/fastchat/serve/openai_api_server.py:506: PydanticDeprecatedSince20: The `dict` method is deprecated; use `model_dump` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
2024-08-16 03:11:13 | ERROR | stderr |   yield f"data: {json.dumps(chunk.dict(exclude_unset=True), ensure_ascii=False)}\n\n"
2024-08-16 03:11:14 | ERROR | stderr | /usr/local/lib/python3.10/site-packages/fastchat/serve/openai_api_server.py:536: PydanticDeprecatedSince20: The `json` method is deprecated; use `model_dump_json` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
2024-08-16 03:11:14 | ERROR | stderr |   yield f"data: {chunk.json(exclude_unset=True, ensure_ascii=False)}\n\n"
2024-08-16 03:11:14 | ERROR | stderr | ERROR:    Exception in ASGI application
2024-08-16 03:11:14 | ERROR | stderr | Traceback (most recent call last):
2024-08-16 03:11:14 | ERROR | stderr |   File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 277, in __call__
2024-08-16 03:11:14 | ERROR | stderr |     await wrap(partial(self.listen_for_disconnect, receive))
2024-08-16 03:11:14 | ERROR | stderr |   File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 273, in wrap
2024-08-16 03:11:14 | ERROR | stderr |     await func()
2024-08-16 03:11:14 | ERROR | stderr |   File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 250, in listen_for_disconnect
2024-08-16 03:11:14 | ERROR | stderr |     message = await receive()
2024-08-16 03:11:14 | ERROR | stderr |   File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 555, in receive
2024-08-16 03:11:14 | ERROR | stderr |     await self.message_event.wait()
2024-08-16 03:11:14 | ERROR | stderr |   File "/usr/local/lib/python3.10/asyncio/locks.py", line 214, in wait
2024-08-16 03:11:14 | ERROR | stderr |     await fut
2024-08-16 03:11:14 | ERROR | stderr | asyncio.exceptions.CancelledError: Cancelled by cancel scope 7f1ffb948160
2024-08-16 03:11:14 | ERROR | stderr | 
2024-08-16 03:11:14 | ERROR | stderr | During handling of the above exception, another exception occurred:
2024-08-16 03:11:14 | ERROR | stderr | 
2024-08-16 03:11:14 | ERROR | stderr |   + Exception Group Traceback (most recent call last):
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
2024-08-16 03:11:14 | ERROR | stderr |   |     result = await app(  # type: ignore[func-returns-value]
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
2024-08-16 03:11:14 | ERROR | stderr |   |     return await self.app(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 292, in __call__
2024-08-16 03:11:14 | ERROR | stderr |   |     await super().__call__(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 122, in __call__
2024-08-16 03:11:14 | ERROR | stderr |   |     await self.middleware_stack(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
2024-08-16 03:11:14 | ERROR | stderr |   |     raise exc
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
2024-08-16 03:11:14 | ERROR | stderr |   |     await self.app(scope, receive, _send)
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 83, in __call__
2024-08-16 03:11:14 | ERROR | stderr |   |     await self.app(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
2024-08-16 03:11:14 | ERROR | stderr |   |     raise exc
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
2024-08-16 03:11:14 | ERROR | stderr |   |     await self.app(scope, receive, sender)
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
2024-08-16 03:11:14 | ERROR | stderr |   |     raise e
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
2024-08-16 03:11:14 | ERROR | stderr |   |     await self.app(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
2024-08-16 03:11:14 | ERROR | stderr |   |     await route.handle(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
2024-08-16 03:11:14 | ERROR | stderr |   |     await self.app(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 69, in app
2024-08-16 03:11:14 | ERROR | stderr |   |     await response(scope, receive, send)
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 270, in __call__
2024-08-16 03:11:14 | ERROR | stderr |   |     async with anyio.create_task_group() as task_group:
2024-08-16 03:11:14 | ERROR | stderr |   |   File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 680, in __aexit__
2024-08-16 03:11:14 | ERROR | stderr |   |     raise BaseExceptionGroup(
2024-08-16 03:11:14 | ERROR | stderr |   | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
2024-08-16 03:11:14 | ERROR | stderr |   +-+---------------- 1 ----------------
2024-08-16 03:11:14 | ERROR | stderr |     | Traceback (most recent call last):
2024-08-16 03:11:14 | ERROR | stderr |     |   File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 273, in wrap
2024-08-16 03:11:14 | ERROR | stderr |     |     await func()
2024-08-16 03:11:14 | ERROR | stderr |     |   File "/usr/local/lib/python3.10/site-packages/starlette/responses.py", line 262, in stream_response
2024-08-16 03:11:14 | ERROR | stderr |     |     async for chunk in self.body_iterator:
2024-08-16 03:11:14 | ERROR | stderr |     |   File "/usr/local/lib/python3.10/site-packages/fastchat/serve/openai_api_server.py", line 536, in chat_completion_stream_generator
2024-08-16 03:11:14 | ERROR | stderr |     |     yield f"data: {chunk.json(exclude_unset=True, ensure_ascii=False)}\n\n"
2024-08-16 03:11:14 | ERROR | stderr |     |   File "/usr/local/lib/python3.10/site-packages/typing_extensions.py", line 2853, in wrapper
2024-08-16 03:11:14 | ERROR | stderr |     |     return arg(*args, **kwargs)
2024-08-16 03:11:14 | ERROR | stderr |     |   File "/usr/local/lib/python3.10/site-packages/pydantic/main.py", line 1011, in json
2024-08-16 03:11:14 | ERROR | stderr |     |     raise TypeError('`dumps_kwargs` keyword arguments are no longer supported.')
2024-08-16 03:11:14 | ERROR | stderr |     | TypeError: `dumps_kwargs` keyword arguments are no longer supported.
2024-08-16 03:11:14 | ERROR | stderr |     +------------------------------------

it looks like chunk.json() functions are using dumps_kwargs keyword arguments which have been abondaned by pydantic (version>2.0.0)
fixed by using json.dumps() to get chunk dict string rather than chunk.json() initialy used.
for example:

# initial
 yield f"data: {chunk.json(exclude_unset=True, ensure_ascii=False)}\n\n"
# after
 yield f"data: {json.dumps(chunk.dict(exclude_unset=True), ensure_ascii=False)}\n\n"   # for pydantic >= 2.0 issue