You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Proposal:
Introduce a ParquetReader in Twister to enable the mapping of Parquet bytes to Java Maps, similar to AvroReader and ProtoReader. This reader will provide a convenient way to read Parquet data and extract it as key-value pairs stored in Maps.
Expected Behavior:
The ParquetReader should allow developers to read Parquet bytes and map the data to Java Maps. Each Parquet row will be represented as a Map object, where column names are used as keys and corresponding values are populated.
Benefits:
Simplified Parquet data processing.
Seamless integration with existing Java data structures.
Enhanced performance and efficiency.
Implementation Considerations:
Utilize existing Parquet libraries, such as Apache Parquet, to handle low-level parsing and decoding. Support various data types defined in the Parquet schema and handle nullable fields appropriately.
Contributor Resources:
Refer to the Twister project's contribution guidelines for instructions on setting up the development environment and submitting a pull request.
Environment:
Twister version: [Specify the version or commit hash]
Operating System: [Specify the OS]
Java version: [Specify the Java version]
Additional environment details: [Provide any relevant details about the environment]
The text was updated successfully, but these errors were encountered:
Issue Description:
Proposal:
Introduce a ParquetReader in Twister to enable the mapping of Parquet bytes to Java Maps, similar to AvroReader and ProtoReader. This reader will provide a convenient way to read Parquet data and extract it as key-value pairs stored in Maps.
Expected Behavior:
The ParquetReader should allow developers to read Parquet bytes and map the data to Java Maps. Each Parquet row will be represented as a Map object, where column names are used as keys and corresponding values are populated.
Benefits:
Implementation Considerations:
Utilize existing Parquet libraries, such as Apache Parquet, to handle low-level parsing and decoding. Support various data types defined in the Parquet schema and handle nullable fields appropriately.
Contributor Resources:
Refer to the Twister project's contribution guidelines for instructions on setting up the development environment and submitting a pull request.
Environment:
The text was updated successfully, but these errors were encountered: