How send tasks in async

[Skorpyon]

Hello!

Pool works perfect.

But I some stuck with solution how to send tasks.

Right now I just sending it in sync manner:

celery_app.send_task("sso.tasks.parse_whitelist", kwargs={"company_id": company.id}

Any sync/async regular ways doesn't works:

# doesn't work
parse_whitelist.apply_async(company_id=company.id)
# I understand that it shouldn't work, but I still tried )))
await parse_whitelist.apply_async(company_id=company.id)

May you advice please?

PS. I've using [email protected]

[the-wondersmith]

@Skorpyon The pool doesn't override anything on the Task class, it just overrides how the workers fulfill them.

All of the usual methods of applying a task (sending it to the broker to be fulfilled by a worker instance) should work as expected, and should still be called synchronously.

[Skorpyon]

should work as expected

Yes, it work. My bad,

parse_whitelist.apply_async(company_id=company.id)

working as expected.

Do you have any ideas what is future of async Task, does Celery team have any plans? I did investigation across it's Github repo but it unclear what is going on with asyncio support. I've tested your pool under small load and it works very well. But sync task sending sure doesn't gave full satisfaction, round trip to RabbitMQ takes near to 50-70 ms, when async dispatcher stop everything. In my situation I send tasks rarely, once per hour, but sure it may exist implementations with dozens tasks per second.

[the-wondersmith]

@Skorpyon Nice! Thank you for the feedback, by the way.

As for async Task, the Celery team has stated several places that asyncio support is a core target for the 6.0 release. No idea what specifically that translates to timeline-wise though.

If it becomes problematic (or there's enough demand and I have the time), I'll add a Task class with await-able dispatch support.

[bartspiering]

+1 for a Task class with await-able dispatch support! That would be very nice to have and I have the feeling that the 6.0 release is not going to be there anytime soon.

[auvipy]

you are right. it will take some time.