Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CELEBORN-1893] Support jemalloc.so.2 #3131

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

helenweng-stripe
Copy link

What changes were proposed in this pull request?

Support libjemalloc.so.2 by default when CELEBORN_PREFER_JEMALLOC is set to true

Why are the changes needed?

Currently only libjemalloc.so is supported. We can manually set CELEBORN_JEMALLOC_PATH to the .so.2 path but would be nice to be able to support this by default. Happy to close this if community prefers just setting CELEBORN_JEMALLOC_PATH manually.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manually tested that the .so.2 path gets set when the .so.2 path doesn't exist and running in prod.

@SteNicholas SteNicholas changed the title CELEBORN-1893 - support jemalloc.so.2 [CELEBORN-1893] Support jemalloc.so.2 Mar 4, 2025
@@ -84,16 +84,19 @@ fi
maybe_enable_jemalloc() {
if [ "${CELEBORN_PREFER_JEMALLOC:-false}" == "true" ]; then
JEMALLOC_PATH="${CELEBORN_JEMALLOC_PATH:-/usr/lib/$(uname -m)-linux-gnu/libjemalloc.so}"
JEMALLOC2_PATH="/usr/lib/$(uname -m)-linux-gnu/libjemalloc.so.2"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
JEMALLOC2_PATH="/usr/lib/$(uname -m)-linux-gnu/libjemalloc.so.2"
JEMALLOC2_PATH="${CELEBORN_JEMALLOC_PATH:-/usr/lib/$(uname -m)-linux-gnu/libjemalloc.so.2}"

@RexXiong
Copy link
Contributor

RexXiong commented Mar 4, 2025

Usually, libjemalloc.so could be linked to libjemalloc.so.2 (correct me if I'm wrong). I'm a bit concerned that there may be other versions of jemalloc in the future.

@pan3793
Copy link
Member

pan3793 commented Mar 4, 2025

The original intention of auto searching jemalloc.so is to simplify the mainstream OS distribution users' configuration effort. Could you please clarify and document inline which OS use /usr/lib/$(uname -m)-linux-gnu/libjemalloc.so.2 by default?

@helenweng-stripe
Copy link
Author

We use it internally, if it's not standardized I am happy to close this PR and explicitly set CELEBORN_JEMALLOC_PATH.

Another fix could be to find any file named libjemalloc.so* and try to use that, what do you think?

@RexXiong
Copy link
Contributor

We use it internally, if it's not standardized I am happy to close this PR and explicitly set CELEBORN_JEMALLOC_PATH.

Another fix could be to find any file named libjemalloc.so* and try to use that, what do you think?

Perhaps we can add another fallback, so that when both the JEMALLOC_PATH and JEMALLOC_FALLBACK do not exist, it can perform an auto search cc @pan3793

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants