Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase LOTUS_FVM_CONCURRENCY #11817

Open
Stebalien opened this issue Apr 3, 2024 · 0 comments · Fixed by filecoin-project/filecoin-ffi#449 · May be fixed by #11867
Open

Increase LOTUS_FVM_CONCURRENCY #11817

Stebalien opened this issue Apr 3, 2024 · 0 comments · Fixed by filecoin-project/filecoin-ffi#449 · May be fixed by #11867

Comments

@Stebalien
Copy link
Member

We set this pretty low (4) because we used to reserve concurrency*1024 instances up-front. We now only allocate 1024 + concurrency * 20, so we should be able to increase the default significantly.

@Stebalien Stebalien added this to FilOz Apr 3, 2024
@Stebalien Stebalien moved this to 📌 Triage in FilOz Apr 3, 2024
Stebalien added a commit to filecoin-project/filecoin-ffi that referenced this issue Apr 3, 2024
We used to allocate 1024 instances per "concurrency" but we now only
allocate 1024 plus 20 per "concurrency". So, I'm increasing the default
and maximum concurrency factors to take advantage of this. Importantly:

1. Max concurrency 128 -> 4096. `1024 + 4096*20 < 128 * 1024`.
2. Default concurrency 4 -> 150. `1024 + 150*20 < 4 * 1024`.

Motivation: I'm hoping that the default (150) is sufficient (more than
the previous max...) so we can just get rid of this knob.

fixes filecoin-project/lotus#11817
Stebalien added a commit to filecoin-project/filecoin-ffi that referenced this issue Apr 6, 2024
We previously used the FVM's `ThreadedExecutor` to execute messages on
separate threads because the FVM requires 64MiB of stack space.

1. The FVM v3 supported for 8 concurrent threads.
2. The FVM v4 supports up to the number of CPU threads available.

Unfortunately, neither version was influenced by the
`LOTUS_FVM_CONCURRENCY` environment variable.

This patch fixes this by:

1. Moving the thread-pool to the FFI itself (sharing it between FVM
versions).
2. Setting the thread-pool size equal to `LOTUS_FVM_CONCURRENCY`.

It also defaults `LOTUS_FVM_CONCURRENCY` to the number of available
CPU threads instead of the previous 4.

fixes filecoin-project/lotus#11817
Stebalien added a commit to filecoin-project/filecoin-ffi that referenced this issue Apr 6, 2024
We previously used the FVM's `ThreadedExecutor` to execute messages on
separate threads because the FVM requires 64MiB of stack space.

1. The FVM v3 supported for 8 concurrent threads.
2. The FVM v4 supports up to the number of CPU threads available.

Unfortunately, neither version was influenced by the
`LOTUS_FVM_CONCURRENCY` environment variable.

This patch fixes this by:

1. Moving the thread-pool to the FFI itself (sharing it between FVM
versions).
2. Setting the thread-pool size equal to `LOTUS_FVM_CONCURRENCY`.

It also defaults `LOTUS_FVM_CONCURRENCY` to the number of available
CPU threads instead of the previous 4.

NOTE: I've also tried increasing the stack size instead of using
threads, but Go _does not_ like it when other foreign mess with the
stack size of _its_ threads (but it has no problem if we create our own
threads).

fixes filecoin-project/lotus#11817
Stebalien added a commit to filecoin-project/filecoin-ffi that referenced this issue Apr 6, 2024
We previously used the FVM's `ThreadedExecutor` to execute messages on
separate threads because the FVM requires 64MiB of stack space.

1. The FVM v3 supported for 8 concurrent threads.
2. The FVM v4 supports up to the number of CPU threads available.

Unfortunately, neither version was influenced by the
`LOTUS_FVM_CONCURRENCY` environment variable.

This patch fixes this by:

1. Moving the thread-pool to the FFI itself (sharing it between FVM
versions).
2. Setting the thread-pool size equal to `LOTUS_FVM_CONCURRENCY`.

It also defaults `LOTUS_FVM_CONCURRENCY` to the number of available
CPU threads instead of the previous 4.

NOTE: I've also tried increasing the stack size instead of using
threads, but Go _does not_ like it when other foreign mess with the
stack size of _its_ threads (but it has no problem if we create our own
threads).

fixes filecoin-project/lotus#11817
@Stebalien Stebalien reopened this Apr 10, 2024
@Stebalien Stebalien linked a pull request Apr 10, 2024 that will close this issue
8 tasks
@rjan90 rjan90 moved this from 📌 Triage to ⌨️In Progress in FilOz Apr 11, 2024
@jennijuju jennijuju added this to the FEVM tech debt milestone Apr 19, 2024
@github-project-automation github-project-automation bot moved this from ⌨️ In Progress to 🎉 Done in FilOz Sep 17, 2024
@rjan90 rjan90 reopened this Sep 17, 2024
@github-project-automation github-project-automation bot moved this from 🎉 Done to 📌 Triage in FilOz Sep 17, 2024
@rjan90 rjan90 moved this from 📌 Triage to ⌨️ In Progress in FilOz Sep 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment