You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was trying to write a read-only ByteBuffer to DynamoDb using the connector but I got
Caused by: java.nio.ReadOnlyBufferException
at java.nio.ByteBuffer.array(ByteBuffer.java:996)
at org.apache.hadoop.dynamodb.DynamoDBUtil.getAttributeSizeBytes(DynamoDBUtil.java:176)
at org.apache.hadoop.dynamodb.DynamoDBUtil.getItemSizeBytes(DynamoDBUtil.java:149)
at org.apache.hadoop.dynamodb.DynamoDBClient.putBatch(DynamoDBClient.java:199)
at org.apache.hadoop.dynamodb.write.AbstractDynamoDBRecordWriter.write(AbstractDynamoDBRecordWriter.java:112)
at org.apache.spark.internal.io.HadoopMapRedWriteConfigUtil.write(SparkHadoopWriter.scala:238)
at org.apache.spark.internal.io.SparkHadoopWriter$$anonfun$4.apply(SparkHadoopWriter.scala:132)
at org.apache.spark.internal.io.SparkHadoopWriter$$anonfun$4.apply(SparkHadoopWriter.scala:129)
at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1439)
at org.apache.spark.internal.io.SparkHadoopWriter$.org$apache$spark$internal$io$SparkHadoopWriter$$executeTask(SparkHadoopWriter.scala:141)
... 10 more
This is very unexpected as the API of AttributeValue().withB() clearly states:
Warning: ByteBuffers returned by the SDK are mutable. Changes to the content or position of the byte buffer will be seen by all objects that have a reference to this object. It is recommended to call ByteBuffer.duplicate() or ByteBuffer.asReadOnlyBuffer() before using or reading from the buffer. This behavior will be changed in a future major version of the SDK.
So the API docs are actually encouraging us to provided a ReadOnly buffer.
The text was updated successfully, but these errors were encountered:
I was trying to write a read-only ByteBuffer to DynamoDb using the connector but I got
This is very unexpected as the API of
AttributeValue().withB()
clearly states:So the API docs are actually encouraging us to provided a ReadOnly buffer.
The text was updated successfully, but these errors were encountered: