Skip to content

[Variant] Add unshred_variant support for Binary and LargeBinary types#9576

Merged
alamb merged 3 commits intoapache:mainfrom
kunalsinghdadhwal:kunal/add-unshred-#9526
Mar 20, 2026
Merged

[Variant] Add unshred_variant support for Binary and LargeBinary types#9576
alamb merged 3 commits intoapache:mainfrom
kunalsinghdadhwal:kunal/add-unshred-#9526

Conversation

@kunalsinghdadhwal
Copy link
Contributor

Which issue does this PR close?

Rationale for this change

shred_variant already supports Binary and LargeBinary types (#9525, #9554), but unshred_variant does not handle these types. This means shredded Binary/LargeBinary columns cannot be converted back to unshredded VariantArrays.

What changes are included in this PR?

Adds unshred_variant support for DataType::Binary and DataType::LargeBinary in parquet-variant-compute/src/unshred_variant.rs:

  • New enum variants PrimitiveBinary and PrimitiveLargeBinary
  • Match arms in append_row and try_new_opt
  • AppendToVariantBuilder impls for BinaryArray and LargeBinaryArray

Are these changes tested?

Yes

Are there any user-facing changes?

No breaking changes

Signed-off-by: Kunal Singh Dadhwal <kunalsinghdadhwal@gmail.com>
Signed-off-by: Kunal Singh Dadhwal <kunalsinghdadhwal@gmail.com>
@github-actions github-actions bot added the parquet-variant parquet-variant* crates label Mar 19, 2026
@kunalsinghdadhwal
Copy link
Contributor Author

Kindly review @sdf-jkl

Copy link
Contributor

@sdf-jkl sdf-jkl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kunalsinghdadhwal! LGTM.

Run cargo fmt to fix lint and we'll ask someone with approval rights to push this forward.

Signed-off-by: Kunal Singh Dadhwal <kunalsinghdadhwal@gmail.com>
@kunalsinghdadhwal
Copy link
Contributor Author

Run cargo fmt to fix lint and we'll ask someone with approval rights to push this forward.

Done

@sdf-jkl
Copy link
Contributor

sdf-jkl commented Mar 19, 2026

@scovich please take a look when available. Thanks!

@alamb alamb changed the title Kunal/add unshred #9526 [Variant] Add unshred_variant support for Binary and LargeBinary types Mar 19, 2026
Copy link
Contributor

@scovich scovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for filling the gap

@alamb alamb merged commit 322f9ce into apache:main Mar 20, 2026
18 checks passed
@alamb
Copy link
Contributor

alamb commented Mar 20, 2026

Thanks @scovich and @sdf-jkl and @kunalsinghdadhwal -- merging to get this into the 58.1.0 release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parquet-variant parquet-variant* crates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Variant] Add unshred_variant support for Binary and LargeBinary types

4 participants