Tokio for compute bound tasks

[tbetcke]

Hi,

I have been trying to figure out how good Tokio is for compute purposes and this resulted in an active Reddit discussion a few days ago.

The main pro and cons arguments seemed to be as follows:

Pro:

• Tokio has a multithreaded work stealing scheduler like Rayon, so in principle should be suited for compute.
• It is possible using async/await to build complex task graphs at runtime, something that Rayon does not allow.
• There are positive user stories, such as this blog post..

Cons:

• Most Tokio use cases are I/O bound. So little experience with the task scheduler for compute purposees (e.g. does it minimize context switches that negatively effect the cache?)
• One should not mix I/O bound and compute bound tasks in the same runtime scheduler (though we are not interested in this as our workload is only compute with all tasks of similar size)
• There are negative user stories, such as this one.

At the moment we are tending towards the decision to stick with Rayon for compute tasks even though this would limit some more complex tasking scenarios.

I'd be very interested to hear more opinions about this topic from the Tokio community.

[Noah-Kennedy]

It certainly can work for this, it's just not what it is designed for. Tokio is designed around the assumption that tasks are going to be generally nonblocking and will yield to the runtime frequently. This assumption is baked into a number of things, such as our parking routine and our workstealing implementation.

I'd generally suggest sticking with Rayon for compute-bound workloads. If you have a workload that is bound by both IO and compute, then I would recommend mixing Tokio with Rayon.

[tbetcke]

Thank you for your response. Do you know of any plans or a project that aims to develop a compute optimised task scheduler for async/await? There are lots of compute applications that would strongly benefit from concurrency directly baked into the programming language. Rayon is great. But its fork/join model is much less flexible than async/await would be.

[Noah-Kennedy]

Not really, but if you need something more complex, you can just use tokio. It will work. It's just not what I would recommend people to go for first.