Support both 0x01 and 0x02 as type for list of booleans in thrift metadata #7052
Conversation
parquet/src/thrift.rs
Outdated
| // Boolean collection type encoded as 0x01, as used by this crate when writing. | ||
| // Values encoded as 1 (true) or 2 (false) as in the current version of the thrift | ||
| // documentation. | ||
| let bytes = vec![25, 33, 2, 1, 25, 8, 25, 8, 21, 0, 0]; |
There was a problem hiding this comment.
Minor nit: I'd find this easier to decode in my head if these were hex values, but that might just be me.
|
Given this reader was copied from the upstream thrift impl, which we still use in some places, I wonder if this change needs to be made there as well? |
You are right, I hoped this might have been fixed already upstream. I'll look into providing a bugfix there too. I think inside arrow-rs the upstream |
Which issue does this PR close?
Rationale for this change
The thrift documentation for lists of booleans was recently updated to clarify encoding of booleans:
At least the go implementation seems to encode the type as 0x02
What changes are included in this PR?
Be a bit more lenient and accept both types.
Are there any user-facing changes?
No, this should be a compatible change. Might even be worth backparting to a bugfix release.