You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[SPARK-52738][SQL] Support aggregating the TIME type with a UDAF when the underlying buffer is an UnsafeRow
### What changes were proposed in this pull request?
- Change `BufferSetterGetterUtils` to use `InternalRow.setLong` for setting TIME values rather then `InternalRow.update`.
- Change `BufferSetterGetterUtils` to use `InternalRow.getLong` for getting TIME values.
- Update the test "udaf with all data types" in `AggregationQuerySuite` so that it checks aggregation with both an unsafe and safe aggregation buffer. Since SPARK-41359, that test has been testing with only a safe aggregation buffer.
### Why are the changes needed?
When a query uses a UDAF to aggregate a TIME column , and all other columns are "mutable" (as determined by `UnsafeRow#isMutable`), the aggregator creates an `UnsafeRow` for the low-level aggregation buffer.
However, the wrapper of that buffer (`MutableAggregationBufferImpl`) fails to properly set up a field setter function for the TIME column, so it attempts to call `UnsafeRow.update` on the underlying buffer. The `UnsafeRow` instance throws `org.apache.spark.SparkUnsupportedOperationException`:
```
Exception in task 0.0 in stage 0.0 (TID 0)
org.apache.spark.SparkUnsupportedOperationException: [UNSUPPORTED_CALL.WITHOUT_SUGGESTION] Cannot call the method "update" of the class "org.apache.spark.sql.catalyst.expressions.UnsafeRow". SQLSTATE: 0A000
```
See SPARK-52738 for a reproduction example.
### Does this PR introduce _any_ user-facing change?
No. The TIME type is not released yet.
### How was this patch tested?
Updated a unit test.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes#51430 from bersprockets/time_udaf.
Authored-by: Bruce Robbins <[email protected]>
Signed-off-by: Max Gekk <[email protected]>
0 commit comments