SocketIO ASGIApp (FastAPI) + Gunicorn Process Manager Example for long running Job #880

Udayaprasad · 2022-02-26T11:24:03Z

Udayaprasad
Feb 26, 2022

ASGI Server App details.

The Server app runs for a long time (Generally 5-10 mins), We don't have the option to use WebSocket protocol due to some restrictions at the customer side, so we have opted SocketIO protocol by maintaining client-specific sid for each request. Our existing FastAPI framework uses Gunicorn to manage concurrent tasks (Inside a POD), due to this long-running time, every subsequent request is going to a different worker context with different sid, so ultimately the client application has no idea about the initially assigned sid.

Due to the above situation, there are multiple duplicate runs that happened on the server-side.

Do you have any example code to work with this? I also see that on client disconnect (Using uvicron server), the server is displaying the disconnected message, but it is not stopping the client request (Thread is running).

Packages used.

python-socketio==5.5.2
fastapi==0.68.1

Code snippet for both Server (Using uvicorn server - for Thread not stopping even after client disconnect problem) and Client.


import uvicorn
import socketio
from fastapi import FastAPI
from fastapi.encoders import jsonable_encoder
from fastapi.responses import HTMLResponse, FileResponse
from fastapi.openapi.utils import get_openapi
from fastapi.middleware.cors import CORSMiddleware
from starlette_context import middleware, plugins

fast_app = FastAPI()

sio = socketio.AsyncServer(async_mode='asgi')

fast_app.include_router(different.router)

fast_app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_methods=["*"],
    allow_headers=["*"],
    allow_credentials=True,
)

fast_app.add_middleware(
    middleware.ContextMiddleware,
    plugins=(
        plugins.ForwardedForPlugin(),
        plugins.UserAgentPlugin()
    ),
)

app = socketio.ASGIApp(sio, other_asgi_app=fast_app)

html = """
<!doctype html>
<html>
  <head>
    <title>Socket.IO Test App</title>
  </head>
  <body>
    <h1>Socket.IO Test App</h1>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/3.0.4/socket.io.min.js"></script>
    <script>
    const sio = io("http://0.0.0.0:5000");

    sio.on('connect', () => {
      console.log('connected');
      sio.emit('clientRequest', {
      "firstProperty": {},
      "referenceProperty": {},
      "restrictions": [],
      "components": []
    }
    );
    });
    
    sio.on('disconnect', () => {
      console.log('disconnected');
    });
    
    sio.on('statusUpdate', (data) => {
      console.log(data);
    });
    
    sio.on('keepAlive', (data) => {
      console.log(data);
      function call_func(){
          setTimeout(waitForCalling, 3000) // Wait 3 seconds before continuing
      }
      function waitForCalling(){
           sio.emit('cancel', {status: "cancel"})
      }
      call_func();
    });
    
    sio.on('close', (data) => {
      console.log(data);
    });
    </script>
  </body>
</html>
"""

@sio.event
async def connect(sid, environ):
    print(sid, 'Connected')


@sio.event
async def disconnect(sid):
    print(sid, 'Disconnected')
    # await sio.disconnect(sid=sid)


@sio.on("clientRequest")
async def server_generate_events(sid, data):
	"""
	A Very long-running job with dynamic message exchange and alose keepAlive notifications
	"""


@fast_app.get("/test",
              response_class=HTMLResponse,
              status_code=200)
async def get():
    return HTMLResponse(html)


if __name__ == '__main__':
    kwargs = {"host": "0.0.0.0", "port": 5000}
    kwargs.update({"debug": True, "reload": True})
    uvicorn.run("main_action:app", **kwargs)

Thanks,
Uday

miguelgrinberg · 2022-02-27T10:16:23Z

miguelgrinberg
Feb 27, 2022
Maintainer

Have you read the documentation? Gunicorn's load balancer is very simplisitic and cannot be configured to work with this package, so you cannot use more then one worker per gunicorn instance.

3 replies

Udayaprasad Feb 28, 2022
Author

Thank you for your answer. I am also facing the issue w.r.t disconnect event, The above code (Server-side) is not stopped running after I close the browser window (disconnect event triggered, but the thread didn't close).

Also, we are moving to Kubernetes-based deployment soon, hope it supports ELB/Ingress-based load balancer.

Thanks,
Uday

miguelgrinberg Feb 28, 2022
Maintainer

I'm sorry, but when I say you should read the documentation I really mean it. The requirements for the load balancer are documented, there is no need to ask me. https://python-socketio.readthedocs.io/en/latest/server.html#scalability-notes

Udayaprasad Feb 28, 2022
Author

The load balancer must be configured to always forward requests from a client to the same worker process. Load balancers call this sticky sessions, or session affinity.

Yes.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SocketIO ASGIApp (FastAPI) + Gunicorn Process Manager Example for long running Job #880

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 1 comment 3 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

SocketIO ASGIApp (FastAPI) + Gunicorn Process Manager Example for long running Job #880

Udayaprasad Feb 26, 2022

Replies: 1 comment · 3 replies

miguelgrinberg Feb 27, 2022 Maintainer

Udayaprasad Feb 28, 2022 Author

miguelgrinberg Feb 28, 2022 Maintainer

Udayaprasad Feb 28, 2022 Author

Udayaprasad
Feb 26, 2022

Replies: 1 comment 3 replies

miguelgrinberg
Feb 27, 2022
Maintainer

Udayaprasad Feb 28, 2022
Author

miguelgrinberg Feb 28, 2022
Maintainer

Udayaprasad Feb 28, 2022
Author