Skip to content

Commit 82439b2

Browse files
authored
Fix pytest hanging when test fails by closing PostgreSQL connection using with statement (#71)
This patch depends on #70. The pytest may hang when some of the test fails. This is a workflow run that exhibited this problem: https://github.com/apache/sedona-db/actions/runs/17668086933/job/50213528921 Pytest does not free up resources held by local variables by calling `__del__` in time when a test fails. When pytest captured the AssertionError, it effectively adds a reference to the local c object, preventing it from being __del__-ed when the function ends. This leaves the transaction started by the failed test active when the next test starts running. We generate test data by dropping a temporary table, inserting data into the temporary table, and finally renaming the temporary table. Dropping the temporary table will be blocked by a lock held by the still-active transaction started by the previously failed test: ``` postgres=# select pid, query, state, wait_event, wait_event_type, backend_type from pg_stat_activity; pid | query | state | wait_event | wait_event_type | backend_type -------+--------------------------------------------------------------------------------------------+---------------------+---------------------+-----------------+------------------------------ 99895 | | | AutoVacuumMain | Activity | autovacuum launcher 99896 | | | LogicalLauncherMain | Activity | logical replication launcher 83813 | select pid, query, state, wait_event, wait_event_type, backend_type from pg_stat_activity; | active | | | client backend 47073 | DROP TABLE IF EXISTS "public" . "sjoin_polygon_xtemp" | idle in transaction | ClientRead | Client | client backend 51565 | DROP TABLE IF EXISTS "public" . "sjoin_point_xtemp" | active | relation | Lock | client backend 66330 | DROP TABLE IF EXISTS "public" . "sjoin_point_xtemp" | active | relation | Lock | client backend 71494 | DROP TABLE IF EXISTS "public" . "sjoin_point_xtemp" | active | relation | Lock | client backend 99892 | | | BgWriterHibernate | Activity | background writer 99891 | | | CheckpointerMain | Activity | checkpointer 99894 | | | WalWriterMain | Activity | walwriter ``` This patch fixes this problem by managing connections using `with` statement. The transaction will be committed or rolled back when the connection is closed. This prevents us from leaking transactions and blocking ourselves on table locks.
1 parent 2be2f35 commit 82439b2

File tree

4 files changed

+666
-637
lines changed

4 files changed

+666
-637
lines changed

python/sedonadb/python/sedonadb/testing.py

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,17 @@ class DBEngine:
6363
of that name.
6464
"""
6565

66+
def close(self):
67+
"""Close the connection - base implementation does nothing"""
68+
pass
69+
70+
def __enter__(self):
71+
return self
72+
73+
def __exit__(self, exc_type, exc_val, exc_tb):
74+
self.close()
75+
return False
76+
6677
@classmethod
6778
def name(cls) -> str:
6879
"""This engine's name
@@ -399,6 +410,11 @@ def __init__(self, uri=None):
399410
uri = "postgresql://localhost:5432/postgres?user=postgres&password=password"
400411
self.con = adbc_driver_postgresql.dbapi.connect(uri)
401412

413+
def close(self):
414+
"""Close the connection"""
415+
if self.con:
416+
self.con.close()
417+
402418
@classmethod
403419
def name(cls):
404420
return "postgis"

0 commit comments

Comments
 (0)