Actors not using multiple CPUs

[wizeman]

As far as I understand, on most actor implementations, different actors can process their own messages in parallel with respect to other actors, which makes things go faster.
But I can't seem to make xactor do that:

use xactor::{message, Actor, Context, Handler, Result};

const ACTORS: usize = 8;

#[message(result = "usize")]
struct Calc;

struct Calculator;

impl Actor for Calculator {}

#[async_trait::async_trait]
impl Handler<Calc> for Calculator {
    async fn handle(&mut self, _ctx: &mut Context<Self>, _msg: Calc) -> usize {
        let mut res: usize = 1;

        for i in 2..100_000_000 {
            res = res.wrapping_mul(i);
            res /= i - 1;
        }

        res
    }
}

#[xactor::main]
async fn main() -> Result<()> {
    let mut children = Vec::with_capacity(ACTORS);
    let mut responses = Vec::with_capacity(ACTORS);

    for _ in 0..ACTORS {
        children.push(Calculator.start().await.unwrap());
    }

    for addr in &children {
        responses.push(addr.call(Calc));
    }

    let mut res = 0;

    for promise in responses {
        res += promise.await.unwrap();
    }

    println!("result: {}", res);

    Ok(())
}

This is what I get:

$ env ASYNC_STD_THREAD_COUNT=8 time -v target/release/xactor
result: 799999992
        Command being timed: "target/release/xactor"
        User time (seconds): 5.58
        System time (seconds): 0.00
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.58
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 2804
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 211
        Voluntary context switches: 51
        Involuntary context switches: 504
        Swaps: 0
        File system inputs: 11
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

Since AFAIU I am sending the messages to all the actors before starting to await their responses, I would have expected CPU usage to be around 800% on my 8-logical-core CPU rather than 99%.

Am I doing something wrong or is this expected?

[wizeman]

For reference, this very similar version of the code which only uses async-std (but not xactor) actually uses all cores in parallel and therefore runs much faster (note: it needs the attributes cargo feature to be enabled for async-std):

use async_std::task;

const ACTORS: usize = 8;

async fn handle() -> usize {
    let mut res: usize = 1;

    for i in 2..100_000_000 {
        res = res.wrapping_mul(i);
        res /= i - 1;
    }

    res
}

#[async_std::main]
async fn main() {
    let mut promises = Vec::with_capacity(ACTORS);

    for _ in 0..ACTORS {
        promises.push(task::spawn(handle()));
    }

    let mut res = 0;

    for promise in promises {
        res += promise.await;
    };

    println!("result: {}", res);
}

Result:

$ env ASYNC_STD_THREAD_COUNT=8 time -v target/release/rust-async
result: 799999992
        Command being timed: "target/release/rust-async"
        User time (seconds): 12.28
        System time (seconds): 0.02
        Percent of CPU this job got: 754%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.63
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 2736
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 209
        Voluntary context switches: 27
        Involuntary context switches: 1838
        Swaps: 0
        File system inputs: 11
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

[joseluis]

I'm also interested in this. I'm searching for a system where you can have some actors running as corutines and some other actors running in their own threads, but I'm still not sure whether this crate would actually help with that.

[sunny-g]

@wizeman I'm able to reproduce what you stated, but also reproduce the inverse of your observed results by changing the first example to

for addr in children.into_iter() {
    responses.push(xactor::spawn(async move {
        addr.call(Calc).await
    }));
}

and the second example to

promises.push(handle());

Not sure how that helps as I'm not too familiar with async/executor internals, but it seems that there are ways of mitigating what you've observed, and coercing lower performance behaviour.