Skip to content
This repository was archived by the owner on Aug 19, 2025. It is now read-only.
This repository was archived by the owner on Aug 19, 2025. It is now read-only.

unable to iterate over multiple rows from database using python in celery #333

@encryptblockr

Description

@encryptblockr

I am using the databases python package (https://pypi.org/project/databases/) to manage connection to my postgresql database

from the documentation (https://www.encode.io/databases/database_queries/#queries)
it says i can either use

# Fetch multiple rows without loading them all into memory at once
query = notes.select()
async for row in database.iterate(query=query):
    ...

or

# Fetch multiple rows
query = notes.select()
rows = await database.fetch_all(query=query)

Here is what i have tried:

def check_all_orders():
    query = "SELECT * FROM orders WHERE shipped=True"
    return database.fetch_all(query)

...
...
...

@app.task
async def check_orders():

    query = await check_all_orders()
    
    today = datetime.utcnow()

    for q in query:
        if q.last_notification is not None:
            if (today - q.last_notification).total_seconds() < q.cooldown:
                continue

and

@app.task
async def check_orders():

    query = "SELECT * FROM orders WHERE shipped=True"

    today = datetime.utcnow()

    async for q in database.iterate(query=query):
        if q.last_notification is not None:
            if (today - q.last_notification).total_seconds() < q.cooldown:
                continue

i have used both but getting the following error:

raise TypeError(f'Object of type {o.class.name} '
kombu.exceptions.EncodeError: Object of type coroutine is not JSON serializable

full error below

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 472, in trace_task
    mark_as_done(
  File "/usr/local/lib/python3.9/site-packages/celery/backends/base.py", line 154, in mark_as_done
    self.store_result(task_id, result, state, request=request)
  File "/usr/local/lib/python3.9/site-packages/celery/backends/base.py", line 434, in store_result
    self._store_result(task_id, result, state, traceback,
  File "/usr/local/lib/python3.9/site-packages/celery/backends/base.py", line 856, in _store_result
    self._set_with_state(self.get_key_for_task(task_id), self.encode(meta), state)
  File "/usr/local/lib/python3.9/site-packages/celery/backends/base.py", line 324, in encode
    _, _, payload = self._encode(data)
  File "/usr/local/lib/python3.9/site-packages/celery/backends/base.py", line 328, in _encode
    return dumps(data, serializer=self.serializer)
  File "/usr/local/lib/python3.9/site-packages/kombu/serialization.py", line 220, in dumps
    payload = encoder(data)
  File "/usr/local/lib/python3.9/contextlib.py", line 135, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.9/site-packages/kombu/serialization.py", line 53, in _reraise_errors
    reraise(wrapper, wrapper(exc), sys.exc_info()[2])
  File "/usr/local/lib/python3.9/site-packages/kombu/exceptions.py", line 21, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.9/site-packages/kombu/serialization.py", line 49, in _reraise_errors
    yield
  File "/usr/local/lib/python3.9/site-packages/kombu/serialization.py", line 220, in dumps
    payload = encoder(data)
  File "/usr/local/lib/python3.9/site-packages/kombu/utils/json.py", line 65, in dumps
    return _dumps(s, cls=cls or _default_encoder,
  File "/usr/local/lib/python3.9/json/__init__.py", line 234, in dumps
    return cls(
  File "/usr/local/lib/python3.9/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/lib/python3.9/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/lib/python3.9/site-packages/kombu/utils/json.py", line 55, in default
    return super().default(o)
  File "/usr/local/lib/python3.9/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
kombu.exceptions.EncodeError: Object of type coroutine is not JSON serializable

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions