GH-48199: [Python][Parquet] Expose existing Parquet C++ metadata about index_page and bloom_filter to Python #48201
+72
−10
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…a about index_page and bloom_filter to Python
|
|
github-actions
bot
added
Component: Python
awaiting committer review
raulcd
commented
Nov 21, 2025
| assert metadata.row_group(0).column(0).has_dictionary_page is True | ||
| assert metadata.row_group(0).column(0).dictionary_page_offset == 4 | ||
| assert metadata.row_group(0).column(0).has_index_page is True | ||
| assert metadata.row_group(0).column(0).index_page_offset == 0 |
Member
Author
There was a problem hiding this comment.
Feels weird that has_index_page is True and index_page_offset is 0 but this is a C++ issue (or a file issue) we are just acting as a pass-through here.
github-actions
bot
added
awaiting changes
Member
Author
|
@github-actions crossbow submit -g python |
|
Revision: 26b9b63 Submitted crossbow builds: ursacomputing/crossbow @ actions-e4a9ff08ad
|
github-actions
bot
added
awaiting merge
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rationale for this change
The Parquet C++ implementation contains information about some metadata that is not exposed to PyArrow. This PR is to expose those.
What changes are included in this PR?
Expose present
parquet::ColumnChunkMetaData:Are these changes tested?
Yes
Are there any user-facing changes?
Yes, new API methods on pyarrow.parquet ColumnChunkMetadata.