Skip to content

Conversation

pitrou
Copy link
Member

@pitrou pitrou commented Sep 17, 2025

Rationale for this change

According to microsoft/mimalloc#1073 , mimalloc v3 is preferred over v2 for production usage.

There are reports of higher than expected memory consumption with mimalloc 2.2.x, notably when reading Parquet data (example: GH-47266).

What changes are included in this PR?

Bump to mimalloc 3.1.5, which is the latest mimalloc 3.1.x release as of this writing.

Are these changes tested?

Yes, by existing tests and CI.

Are there any user-facing changes?

Hopefully not, besides a potential reduction in memory usage due to improvements in mimalloc v3.

@github-actions github-actions bot added the awaiting review Awaiting review label Sep 17, 2025
@pitrou
Copy link
Member Author

pitrou commented Sep 17, 2025

@github-actions crossbow submit -g cpp

This comment was marked as outdated.

@pitrou
Copy link
Member Author

pitrou commented Sep 18, 2025

@github-actions crossbow submit -g cpp

This comment was marked as outdated.

@pitrou
Copy link
Member Author

pitrou commented Sep 18, 2025

@github-actions crossbow submit -g cpp

This comment was marked as outdated.

@pitrou
Copy link
Member Author

pitrou commented Sep 18, 2025

@ursabot please benchmark

@voltrondatabot
Copy link

Benchmark runs are scheduled for commit 709942b. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.

Copy link

Thanks for your patience. Conbench analyzed the 4 benchmarking runs that have been run so far on PR commit 709942b.

There were 80 benchmark results indicating a performance regression:

The full Conbench report has more details.

@pitrou
Copy link
Member Author

pitrou commented Sep 22, 2025

@github-actions crossbow submit -g cpp

This comment was marked as outdated.

@pitrou
Copy link
Member Author

pitrou commented Oct 7, 2025

@github-actions crossbow submit -g cpp

@pitrou pitrou changed the title GH-47588: [C++] Bump mimalloc version to 3.0.9 GH-47588: [C++] Bump mimalloc version to 3.1.5 Oct 7, 2025

This comment was marked as outdated.

@github-actions github-actions bot added awaiting committer review Awaiting committer review awaiting changes Awaiting changes and removed awaiting review Awaiting review awaiting committer review Awaiting committer review awaiting changes Awaiting changes labels Oct 7, 2025
@pitrou
Copy link
Member Author

pitrou commented Oct 8, 2025

@github-actions crossbow submit -g cpp -g linux -g r

@pitrou
Copy link
Member Author

pitrou commented Oct 8, 2025

@github-actions crossbow submit cp313

Copy link

github-actions bot commented Oct 8, 2025

Invalid group(s) {'linux'}. Must be one of {'verify-rc-source-linux', 'homebrew', 'r', 'fuzz', 'c-glib', 'ruby', 'example-cpp', 'verify-rc-source-macos', 'conan', 'cpp', 'wheel', 'vcpkg', 'nightly', 'nightly-tests', 'verify-rc', 'python', 'nightly-packaging', 'test', 'verify-rc-source', 'example-python', 'packaging', 'cuda', 'nightly-release', 'integration', 'example'}
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/18336337327

This comment was marked as outdated.

@pitrou
Copy link
Member Author

pitrou commented Oct 8, 2025

@github-actions crossbow submit -g cpp -g r

@pitrou pitrou added the CI: Extra Run extra CI label Oct 8, 2025

This comment was marked as outdated.

@pitrou
Copy link
Member Author

pitrou commented Oct 8, 2025

CI failures are unrelated. Should we merge this @kou @WillAyd ?

Copy link
Member

@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@pitrou
Copy link
Member Author

pitrou commented Oct 9, 2025

@github-actions crossbow submit -g cpp -g r cp313

Copy link

github-actions bot commented Oct 9, 2025

Revision: 4294e34

Submitted crossbow builds: ursacomputing/crossbow @ actions-562e16c367

Task Status
example-cpp-minimal-build-static GitHub Actions
example-cpp-minimal-build-static-system-dependency GitHub Actions
example-cpp-tutorial GitHub Actions
r-binary-packages GitHub Actions
r-recheck-most GitHub Actions
test-build-cpp-fuzz GitHub Actions
test-conda-cpp GitHub Actions
test-conda-cpp-valgrind GitHub Actions
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1 GitHub Actions
test-debian-12-cpp-amd64 GitHub Actions
test-debian-12-cpp-i386 GitHub Actions
test-fedora-42-cpp GitHub Actions
test-r-arrow-backwards-compatibility GitHub Actions
test-r-depsource-bundled Azure
test-r-depsource-system GitHub Actions
test-r-dev-duckdb GitHub Actions
test-r-devdocs GitHub Actions
test-r-extra-packages GitHub Actions
test-r-gcc-11 GitHub Actions
test-r-gcc-12 GitHub Actions
test-r-install-local GitHub Actions
test-r-install-local-minsizerel GitHub Actions
test-r-linux-as-cran GitHub Actions
test-r-linux-rchk GitHub Actions
test-r-linux-sanitizers GitHub Actions
test-r-linux-valgrind GitHub Actions
test-r-m1-san GitHub Actions
test-r-macos-as-cran GitHub Actions
test-r-minimal-build Azure
test-r-offline-maximal GitHub Actions
test-r-offline-minimal Azure
test-r-rhub-debian-gcc-devel-lto-latest Azure
test-r-rhub-ubuntu-gcc12-custom-ccache Azure
test-r-rhub-ubuntu-release-latest Azure
test-r-rocker-r-ver-latest Azure
test-r-rstudio-r-base-4.1-focal Azure
test-r-rstudio-r-base-4.2-focal Azure
test-r-ubuntu-22.04 GitHub Actions
test-r-versions GitHub Actions
test-ubuntu-22.04-cpp GitHub Actions
test-ubuntu-22.04-cpp-20 GitHub Actions
test-ubuntu-22.04-cpp-bundled GitHub Actions
test-ubuntu-22.04-cpp-emscripten GitHub Actions
test-ubuntu-22.04-cpp-no-threading GitHub Actions
test-ubuntu-24.04-cpp GitHub Actions
test-ubuntu-24.04-cpp-bundled-offline GitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundled GitHub Actions
test-ubuntu-24.04-cpp-gcc-14 GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formats GitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizer GitHub Actions
wheel-macos-monterey-cp313-cp313-amd64 GitHub Actions
wheel-macos-monterey-cp313-cp313-arm64 GitHub Actions
wheel-macos-monterey-cp313-cp313t-amd64 GitHub Actions
wheel-macos-monterey-cp313-cp313t-arm64 GitHub Actions
wheel-manylinux-2-28-cp313-cp313-amd64 GitHub Actions
wheel-manylinux-2-28-cp313-cp313-arm64 GitHub Actions
wheel-manylinux-2-28-cp313-cp313t-amd64 GitHub Actions
wheel-manylinux-2-28-cp313-cp313t-arm64 GitHub Actions
wheel-musllinux-1-2-cp313-cp313-amd64 GitHub Actions
wheel-musllinux-1-2-cp313-cp313-arm64 GitHub Actions
wheel-musllinux-1-2-cp313-cp313t-amd64 GitHub Actions
wheel-musllinux-1-2-cp313-cp313t-arm64 GitHub Actions
wheel-windows-cp313-cp313-amd64 GitHub Actions
wheel-windows-cp313-cp313t-amd64 GitHub Actions

@pitrou pitrou merged commit e1f727c into apache:main Oct 9, 2025
60 checks passed
@pitrou pitrou removed the awaiting merge Awaiting merge label Oct 9, 2025
@pitrou pitrou deleted the gh-bump-mimalloc branch October 9, 2025 09:47
raulcd pushed a commit that referenced this pull request Oct 9, 2025
### Rationale for this change

According to microsoft/mimalloc#1073 , mimalloc v3 is preferred over v2 for production usage.

There are reports of higher than expected memory consumption with mimalloc 2.2.x, notably when reading Parquet data (example: GH-47266).

### What changes are included in this PR?

Bump to mimalloc 3.1.5, which is the latest mimalloc 3.1.x release as of this writing.

### Are these changes tested?

Yes, by existing tests and CI.

### Are there any user-facing changes?

Hopefully not, besides a potential reduction in memory usage due to improvements in mimalloc v3.

* GitHub Issue: #47588

Authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
Copy link

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit e1f727c.

There were 5 benchmark results indicating a performance regression:

The full Conbench report has more details. It also includes information about 16 possible false positives for unstable benchmarks that are known to sometimes produce them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants