Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

high memory consumption for producer-consumer layouts with default settings #56

Open
carns opened this issue Apr 24, 2018 · 0 comments
Open

Comments

@carns
Copy link
Contributor

carns commented Apr 24, 2018

If one ES consistently launches ULTs to execute on a different slave ES (for example, to offload specific calculations or I/O operations), then the per-ES stack queue on the latter will eventually fill up and consume hundreds of MiB of RAM. If that ES never launches it's own ULTs then those entries will not be used and the queue will never be emptied.

The workaround is to set the ABT_MEM_MAX_NUM_STACKS environment variable to a low value (it defaults to 65536) to limit how many entries will be saved in the per-ES queue. This is done automatically in the margo library: https://xgitlab.cels.anl.gov/sds/margo/blob/master/src/margo.c#L195

See https://lists.argobots.org/pipermail/discuss/2018-February/000020.html for discussion, just entering on github so that we don't lose track of the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant