-
Notifications
You must be signed in to change notification settings - Fork 3.4k
Use bulk BlockBuilder APIs for appending range of blocks #26964
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Reviewer's GuideThis PR promotes the internal bulk copy method appendRawBlockRange to a public BlockBuilder API and refactors array and map code to use this API instead of manual loops over ValueBlock. Class diagram for BlockBuilder and related block types after appendRawBlockRange promotionclassDiagram
class BlockBuilder {
+newBlockBuilderLike(blockBuilderStatus)
+appendRawBlockRange(rawBlock, offset, length, blockBuilder)
}
class RunLengthEncodedBlock
class DictionaryBlock
class ValueBlock
BlockBuilder <|.. RunLengthEncodedBlock
BlockBuilder <|.. DictionaryBlock
BlockBuilder <|.. ValueBlock
BlockBuilder o-- "appendRawBlockRange()" Block
RunLengthEncodedBlock <|-- Block
DictionaryBlock <|-- Block
ValueBlock <|-- Block
Class diagram for ArrayConcatUtils after refactor to use appendRawBlockRangeclassDiagram
class ArrayConcatUtils {
+appendElement(Type, Block, long)
+appendElement(Type, Block, boolean)
+appendElement(Type, Block, double)
+appendElement(Type, Block, Slice)
+appendElement(Type, Block, Object)
+prependElement(Type, Slice, Block)
+prependElement(Type, Object, Block)
+prependElement(Type, long, Block)
+prependElement(Type, boolean, Block)
+prependElement(Type, double, Block)
}
ArrayConcatUtils --> BlockBuilder : uses
ArrayConcatUtils --> Block : uses
ArrayConcatUtils --> Type : uses
ArrayConcatUtils ..> "appendRawBlockRange()" BlockBuilder
Class diagram for ArrayType and MapType writeObject refactorclassDiagram
class ArrayType {
+writeObject(BlockBuilder, Object)
}
class MapType {
+writeObject(BlockBuilder, Object)
}
ArrayType ..> "appendRawBlockRange()" BlockBuilder : uses
MapType ..> "appendRawBlockRange()" BlockBuilder : uses
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey there - I've reviewed your changes and they look great!
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location> `core/trino-spi/src/main/java/io/trino/spi/block/BlockBuilder.java:95-99` </location>
<code_context>
return newBlockBuilderLike(calculateBlockResetSize(getPositionCount()), blockBuilderStatus);
}
+
+ static void appendRawBlockRange(Block rawBlock, int offset, int length, BlockBuilder blockBuilder)
+ {
+ switch (rawBlock) {
+ case RunLengthEncodedBlock rleBlock -> blockBuilder.appendRepeated(rleBlock.getValue(), 0, length);
+ case DictionaryBlock dictionaryBlock -> blockBuilder.appendPositions(dictionaryBlock.getDictionary(), dictionaryBlock.getRawIds(), offset, length);
</code_context>
<issue_to_address>
**suggestion:** Consider handling unexpected Block subtypes in appendRawBlockRange.
Adding a default case to the switch statement that throws an exception will ensure that any unsupported Block subtype is caught immediately, improving debuggability and maintainability.
```suggestion
switch (rawBlock) {
case RunLengthEncodedBlock rleBlock -> blockBuilder.appendRepeated(rleBlock.getValue(), 0, length);
case DictionaryBlock dictionaryBlock -> blockBuilder.appendPositions(dictionaryBlock.getDictionary(), dictionaryBlock.getRawIds(), offset, length);
case ValueBlock valueBlock -> blockBuilder.appendRange(valueBlock, offset, length);
default -> throw new IllegalArgumentException("Unsupported Block subtype: " + rawBlock.getClass().getName());
}
```
</issue_to_address>
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
core/trino-spi/src/main/java/io/trino/spi/block/BlockBuilder.java
Outdated
Show resolved
Hide resolved
return newBlockBuilderLike(calculateBlockResetSize(getPositionCount()), blockBuilderStatus); | ||
} | ||
|
||
static void appendRawBlockRange(Block rawBlock, int offset, int length, BlockBuilder blockBuilder) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the other PR, I mentioned that I wasn't sure if we want to make this public. My reson for this is because I can't think of a good name for this funcetion. My thought is that this should just be a default method on the interface and not a static method, but we already used the name appendRange for value blocks, which should be preferred as they are way faster. Since ValueBlock extends from Block we don't wan to have the same name as it creates overloading confustion in performance sensitive code. I guess we could go with appendRawRange
, but I don't really like that. Maybe someone else has an idea here. Regardless this method should have javadoc explaining that the other methods are preferred (unless your code will just have the same switch statement)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I updated this to be default void appendBlockRange(Block rawBlock, int offset, int length)
and added javadoc.
PTAL
Allows bulk BlockBuilder APIs to be used more easily
93282ba
to
964ede0
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Expose a public bulk appendBlockRange API on BlockBuilder and replace manual element-copy loops with this method for arrays, maps, and related scalar functions.
- Introduces BlockBuilder.appendBlockRange default method supporting ValueBlock, DictionaryBlock, and RunLengthEncodedBlock.
- Replaces prior private utility (BlockUtil.appendRawBlockRange) and manual loops in array/map builders and functions with appendBlockRange.
- Cleans up imports and removes now-unneeded helper logic.
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
File | Description |
---|---|
MapType.java | Replaces manual key/value loop with appendBlockRange for map entry construction. |
ArrayType.java | Simplifies array writeObject logic using appendBlockRange. |
MapBlockBuilder.java | Switches from BlockUtil helper to new BlockBuilder.appendBlockRange for key/value appends. |
BlockUtil.java | Removes obsolete appendRawBlockRange helper. |
BlockBuilder.java | Adds new public default appendBlockRange method with pattern matching over Block subtypes. |
ArrayBlockBuilder.java | Uses appendBlockRange for element copying in append and appendRange. |
ArrayFlattenFunction.java | Replaces inner element loop with appendBlockRange. |
ArrayConcatUtils.java | Consolidates repeated element-copy loops to appendBlockRange across append/prepend helpers. |
ArrayConcatFunction.java | Uses appendBlockRange for concatenating arrays. |
MergePages.java | Replaces custom appendRawBlock logic with appendBlockRange for page merging. |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Description
Moves
appendRawBlockRange
to BlockBuilder to make it public.Used it in Array operators
Additional context and related issues
Release notes
(x) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:
Summary by Sourcery
Expose bulk block appending in BlockBuilder and update array/map builders and related functions to use the new API for more concise block range appends.
New Features:
Enhancements: