We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
we sometimes run into an exception when closing a ParquetWriter instance:
2024-06-10 10:44:01.398 org.apache.parquet.util.AutoCloseables$ParquetCloseResourceException: Unable to close resource 2024-06-10 10:44:01.398 at org.apache.parquet.util.AutoCloseables.uncheckedClose(AutoCloseables.java:85) 2024-06-10 10:44:01.398 at org.apache.parquet.util.AutoCloseables.uncheckedClose(AutoCloseables.java:94) 2024-06-10 10:44:01.398 at org.apache.parquet.hadoop.InternalParquetRecordWriter.close(InternalParquetRecordWriter.java:144) 2024-06-10 10:44:01.398 at org.apache.parquet.hadoop.ParquetWriter.close(ParquetWriter.java:437) 2024-06-10 10:44:01.398 Caused by: java.nio.channels.ClosedChannelException: null 2024-06-10 10:44:01.398 at org.apache.hadoop.hdfs.ExceptionLastSeen.throwException4Close(ExceptionLastSeen.java:73) 2024-06-10 10:44:01.398 at org.apache.hadoop.hdfs.DFSOutputStream.checkClosed(DFSOutputStream.java:158) 2024-06-10 10:44:01.398 at org.apache.hadoop.hdfs.DFSOutputStream.flushOrSync(DFSOutputStream.java:639) 2024-06-10 10:44:01.398 at org.apache.hadoop.hdfs.DFSOutputStream.hflush(DFSOutputStream.java:594) 2024-06-10 10:44:01.398 at org.apache.hadoop.fs.FSDataOutputStream.hflush(FSDataOutputStream.java:136) 2024-06-10 10:44:01.398 at org.apache.parquet.hadoop.util.HadoopPositionOutputStream.close(HadoopPositionOutputStream.java:65) 2024-06-10 10:44:01.398 at org.apache.parquet.hadoop.ParquetFileWriter.close(ParquetFileWriter.java:1663) 2024-06-10 10:44:01.398 at org.apache.parquet.util.AutoCloseables.close(AutoCloseables.java:49) 2024-06-10 10:44:01.398 at org.apache.parquet.util.AutoCloseables.uncheckedClose(AutoCloseables.java:83)
Reporter: Roelof Naude
Note: This issue was originally created as PARQUET-2496. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered:
No branches or pull requests
we sometimes run into an exception when closing a ParquetWriter instance:
Reporter: Roelof Naude
Note: This issue was originally created as PARQUET-2496. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: