Refactor database session handling to async with retry logic, remove middleware

[ahdamin]

The PR refactors database session management to improve the performance and avoid continuous PendingRollbackError raised after the loss of database connection Database connection error: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query')

Context & design choices

Since genotype-api utilizes FastAPI, which is asynchronous by design, that makes it a bit sensitive to session sharing issues. Multiple requests in an asynchronous environment can overlap and potentially share the session and may lead to issues such as PendingRollbackError.
The dependency injection for database sessions was enough for FastAPI. In Flask, middleware with a global session works because each request runs in its own thread, and the session is scoped to that thread.

When using SQLAlchemy's AsyncSession, Lazy Loading breaks because it does not natively use await, and you may encounter MissingGreenlet. The eager loading was implemented using the selectinload method to avoid the error, without affecting the logic of the service/endpoint.

In some cases, when implementing async queries, an explicit join condition was added because the code switched from accessing the object’s relationship to using a query.

Fixed

Handles lost database connection

[ChrOertlin]

Thanks for the in depth explanation, I learned from that!

Just curious about the async and await propagation. Do we need that throughout the whole code or is it enough to have it in the api when it calls the top level function of a multi layer call chain?

I.e. would this be enough?

async def get_something():
       await do_the_get()
       return

## the do the get
def do_the_get():
      return the_gotten

instead of also having the async and await in the do_the_get()

Edit: I asked GPT seems to be dependent on the underlying function and whether their operation are synchronous or asynchronous of nature.

[ahdamin]

Just curious about the async and await propagation. Do we need that throughout the whole code or is it enough to have it in the api when it calls the top level function of a multi layer call chain?

I.e. would this be enough?

async def get_something():
       await do_the_get()
       return

## the do the get
def do_the_get():
      return the_gotten

instead of also having the async and await in the do_the_get()

Edit: I asked GPT seems to be dependent on the underlying function and whether their operation are synchronous or asynchronous of nature.

I understand your point, and it would be simpler to use only async and await at the API level. However, as you mentioned, it depends on the nature of each function. If a function performs async operations, then both the function itself and any functions calling it need to be marked async and use await.

[Vince-janv]

I understand your point, and it would be simpler to use only async and await at the API level. However, as you mentioned, it depends on the nature of each function. If a function performs async operations, then both the function itself and any functions calling it need to be marked async and use await.

I'm not convinced adding async/await to each function and return is necessary. It feels weird that to use FastAPI one needs to drastically change the syntax in every python function. Should it not be added only to the endpoints which take a lot of time?

[ahdamin]

I'm not convinced adding async/await to each function and return is necessary. It feels weird that to use FastAPI one needs to drastically change the syntax in every python function. Should it not be added only to the endpoints which take a lot of time?

Of course, it is not necessary to change every Python function. Async operations are related to IO operations such as database operations, file operations, etc. Some functions were not changed here.
Can we go for a synchronous FastAPI application? Of course, we can! The question is why?

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud