performance of a lot of short-alive tasks

[wangjia184]

I am using tokio to spawn a lot of tasks(e.g. 30000 / second).
The tasks do not perform I/O operation nor blocking operation.
Each task await on a JoinHandle of another task, or maybe a oneshot::Receiver signaled by another task.
When the waited JoinHandle or oneshot::Receiver is signaled, the task handle the logic and then exits.

The flame graph shows that a lot of time is spent on parking_lot stuff.
The full source code can be found here.

I am asking here if I can optimize the code somehow.

Is that normal parking_lot takes such a lot of time in flame graph?

Would it be better if I avoid spawn so many tasks? e.g. using FuturesUnordered to watch on the oneshot::Receiver, and having a fixed number of tasks executing the code in queue after they are signaled.

[Darksonn]

It seems like all the tasks you spawn will wait on some previous oneshot, essentially making all the tasks run sequentially. It seems to me that you should be able to optimize this if you spawn a single task for handling all the operations. You could think of it as an actor and send it new tasks through an mpsc channel.

[wangjia184]

Thanks @Darksonn , I see you again, on users.rust-lang.org and also here :) you are always so helpful.
The challenge is that -- the waited JoinHandle and Oneshot could be joined. It is not always a single chained actions.
Sometime they could join.

Thanks for the hint, I will take a look at the actor.

[Darksonn]

Yes, but I would still try to find some way to make it work in a single (or few) tasks. If you spawn that many of them and they all behave like this, it will be pretty inefficient. Using a FuturesUnordered may be better, but I think you could beat that too.

[wangjia184]

I tried to change the code to take use of FutureUnordered with a fixed number of tasks to handle them.
https://github.com/wangjia184/rholang-rust/blob/jerry/another_attempt/rho_runtime/src/storage/runner.rs#L125-L147

The test result shows it is ~3 times slower.
The flame graph does not change much, but the oneshot send/receive cost increased a lot.