Bug: SurrealDB destroys Python Async runtime

[dartt0n]

Describe the bug

The library code contains a very (very!) A dangerous call:

surrealdb.py/surrealdb/connection_ws.py

Line 112 in d72ffcc

asyncio.get_event_loop().stop()

Whenever surrealdb doesn't receive a websocket update (e.g., a network issue, an invalid address, a serialization error, etc.), it destroys the current async loop (which, in 99.9% of cases, is the main async event loop). This behavior leads to:

1. Destroyed runtime (asyncio drops every pending/running/awaiting coroutine)
2. Python runtime will fail with a very ambiguous exception:

Traceback (most recent call last):
  File ".../main.py", line 136, in <module>
    asyncio.run(main())
  File ".../python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File ".../python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../python3.12/asyncio/base_events.py", line 683, in run_until_complete
    raise RuntimeError('Event loop stopped before Future completed.')
RuntimeError: Event loop stopped before Future completed.

This exception and traceback do not contain any information about the reason for failure, nor information about the cause, which is the surrealdb library.

Steps to reproduce

Any websocket failure in any async runtime

Expected behaviour

Graceful failure (raise exception / reconnect / etc)

SurrealDB version

2.1.4

surrealdb.py version

0.4.1

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

[remade]

@dartt0n asyncio.CancelledError is thrown when there is a cancel request in the current loop which is the desired effect but I understand the undesirable behaviour when other errors occur (a network issue, an invalid address, a serialization error, etc). An error response can be sent to the listen queue or a dedicated error queue. Gravitating towards the former. The except block for all non asyncio.CancelledError will send the error response to the queue and then continue the loop.

[dartt0n]

asyncio.CancelledError is thrown when there is a cancel request in the current loop which is the desired effect

Destroying the running event loop in a library's code is an unacceptable and unpredictable side effect. This effect will result in closing the asyncio event loop, making any interaction with it (spawning new coroutines, awaiting old ones, etc) throw an exception. There is code to understand how dangerous this is.

import asyncio

async def long_running_transaction():
    try:
        print("starting long running transaction")
        await asyncio.sleep(10)
    except asyncio.CancelledError:
        print("long running transaction cancelled, executing rollback")
        await asyncio.sleep(1)
        print("rollback done")
    else:
        print("long running transaction finished, executing commit")
        await asyncio.sleep(1)
        print("commit done")
    finally:
        print("long running transaction done, closing connection")
        await asyncio.sleep(1)
        print("connection closed")


async def dummy_surreal_example():
    while True:
        try:
            await asyncio.sleep(1)
        except asyncio.CancelledError:
            print("cancelled")
            break

    asyncio.get_event_loop().stop() # <- the root of all evil, hidden inside library code


async def main():
    # spawn several tasks
    task1 = asyncio.create_task(long_running_transaction())
    task2 = asyncio.create_task(dummy_surreal_example())

    await asyncio.sleep(2)
    task2.cancel()  # cancel the task due to some business logic condition, e.g. timeout
    await task2  # wait till graceful shutdown

    # start another task, e.g. cleanup job
    # however, this task will never be spawned (and executed) since runtime is destroyed by `get_event_loop().stop()`
    # you can image this call being in `except` or `finally` block and running some essential transaction logic, e.g. commit or updating customer balance
    task3 = asyncio.create_task(long_running_transaction())
    await task3 


if __name__ == "__main__":
    asyncio.run(main())

[exius-netrunner]

Yup, I get a ton of random errors relating to this all the time:

RuntimeError: Event loop stopped before Future completed.