Skip to content

[fix][offload] Close all resources in BlobStoreBackedReadHandleImplV2.closeAsync#25296

Open
iamsanjaymalakar wants to merge 2 commits intoapache:masterfrom
iamsanjaymalakar:master
Open

[fix][offload] Close all resources in BlobStoreBackedReadHandleImplV2.closeAsync#25296
iamsanjaymalakar wants to merge 2 commits intoapache:masterfrom
iamsanjaymalakar:master

Conversation

@iamsanjaymalakar
Copy link

@iamsanjaymalakar iamsanjaymalakar commented Mar 7, 2026

Motivation

BlobStoreBackedReadHandleImplV2#closeAsync() closes indices and dataStreams sequentially inside a single try/catch. If closing one resource throws, the method exits early and the remaining resources are never closed. This can leak file descriptors and leave resources open when offload read handles are closed.

Modifications

  • Update closeAsync() to attempt closing all OffloadIndexBlockV2 instances and all DataInputStreams.

  • Collect the first IOException and attach subsequent failures via suppressed exceptions.

  • Complete the returned CompletableFuture exceptionally if any close operation fails; otherwise mark the handle Closed and complete normally.

  • doc

  • doc-required

  • doc-not-needed

  • doc-complete

@github-actions
Copy link

github-actions bot commented Mar 7, 2026

@iamsanjaymalakar Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

@github-actions github-actions bot added doc-not-needed Your PR changes do not impact docs and removed doc-label-missing labels Mar 7, 2026
@codelipenghui codelipenghui added this to the 4.2.0 milestone Mar 9, 2026
@codelipenghui codelipenghui added release/4.1.4 release/4.0.10 type/bug The PR fixed a bug or issue reported a bug labels Mar 9, 2026
Copy link
Contributor

@codelipenghui codelipenghui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And could you please also add tests to cover the changes from this PR, which can confirm the issue can be reproduced and avoid regression in the future.

@iamsanjaymalakar
Copy link
Author

Thanks for the reviews! I addressed the feedback by making closeAsync() close all resources without early exit and ensuring the returned future is always completed (even on runtime exceptions). I didn’t add a test because resource-leak/file-lock tests tend to be nondeterministic and flaky across OS/CI environments.

@lhotari
Copy link
Member

lhotari commented Mar 9, 2026

Please run "Personal CI" to validate such changes on your side. You can run Pulsar CI in your personal fork by following these instructions: https://pulsar.apache.org/contribute/personal-ci/. It's free.
The benefit is that you have full control of CI and you won't need to wait for maintainers to approve the CI workflow run to get CI feedback.

@lhotari
Copy link
Member

lhotari commented Mar 9, 2026

One detail about BlobStoreBackedReadHandleImplV2 is that it's not used in Pulsar's production code.
The
org.apache.bookkeeper.mledger.LedgerOffloader#readOffloaded(long, org.apache.bookkeeper.mledger.proto.MLDataFormats.OffloadContext, java.util.Map<java.lang.String,java.lang.String>) method is not called from production code which would use BlobStoreBackedReadHandleImplV2.
This something that I noticed while fixing a flaky tests a while ago in #24331.
"PIP 76: Streaming Offload" hasn't been fully implemented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-not-needed Your PR changes do not impact docs release/4.0.10 release/4.1.4 type/bug The PR fixed a bug or issue reported a bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants