Skip to content

fix: check for key retriever in Parquet decryption properties conversion - #24866

Merged
kosiew merged 3 commits into
apache:mainfrom
adamreeve:key-retriever-check
Sep 10, 2026
Merged

fix: check for key retriever in Parquet decryption properties conversion#24866
kosiew merged 3 commits into
apache:mainfrom
adamreeve:key-retriever-check

Conversation

@adamreeve

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

Previously an error when retrieving the footer key was used as an indication that a key retriever was used, but this isn't reliable. A key retriever implementation might choose to return a footer key even if no key metadata is provided. This could mean that a footer key is set in the converted decryption properties but column keys are not correctly converted.

arrow-rs didn't previously expose whether a key retriever is used, but this was added in 59.0.0.

What changes are included in this PR?

Use the new FileDecryptionProperties::uses_key_retriever to determine whether FileDecryptionProperties uses a key retriever, rather than rely on getting an error when getting the footer key.

What is the testing strategy for this PR?

Updated existing unit test.

Are there any user-facing changes?

No, this changes an error message but should otherwise not affect users except if they had a key retriever implementation that would return footer keys without any metadata being provided.

@github-actions github-actions Bot added the common Related to common crate label Sep 1, 2026
@codecov-commenter

codecov-commenter commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 76.92308% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.72%. Comparing base (1b6dc92) to head (8208093).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/common/src/config.rs 76.92% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24866      +/-   ##
==========================================
- Coverage   81.72%   81.72%   -0.01%     
==========================================
  Files        1127     1127              
  Lines      416310   416312       +2     
  Branches   416310   416312       +2     
==========================================
- Hits       340224   340212      -12     
- Misses      56095    56108      +13     
- Partials    19991    19992       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@adamreeve
adamreeve marked this pull request as ready for review September 2, 2026 00:08

@kosiew kosiew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adamreeve,

Thanks for working on this. The explicit uses_key_retriever() check looks like the right way to handle this. I just have one suggestion to make the regression test cover the original failure mode more directly.

Comment thread datafusion/common/src/config.rs Outdated
@adamreeve

Copy link
Copy Markdown
Contributor Author

Thanks for reviewing this @kosiew!

@kosiew

kosiew commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

🚀
@adamreeve
Thank you for your contribution.

@kosiew
kosiew added this pull request to the merge queue Sep 9, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to no response for status checks Sep 9, 2026
@kosiew
kosiew added this pull request to the merge queue Sep 10, 2026
Merged via the queue into apache:main with commit 06dd627 Sep 10, 2026
38 checks passed
@adamreeve
adamreeve deleted the key-retriever-check branch September 10, 2026 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

common Related to common crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve conversion from FileDecryptionProperties to ConfigFileDecryptionProperties

3 participants