[core] Support nested-key-null-strategy in FieldNestedUpdateAgg operator#8374
[core] Support nested-key-null-strategy in FieldNestedUpdateAgg operator#8374PyRSA wants to merge 6 commits into
Conversation
…nt nested keys in FieldNestedUpdateAgg
…nt nested keys in FieldNestedUpdateAgg
…nt nested keys in FieldNestedUpdateAgg
…-null-strategy' into feature/nested-update-nested-key-null-strategy
| return options.get( | ||
| key(FIELDS_PREFIX + "." + fieldName + "." + NESTED_KEY_NULL_STRATEGY) | ||
| .enumType(NestedKeyNullStrategy.class) | ||
| .noDefaultValue()); |
There was a problem hiding this comment.
Why not create a default value?
There was a problem hiding this comment.
Thanks for the suggestion! I originally avoided a default value because I wanted to distinguish between an unspecified option and an explicitly configured MERGE, so that I could validate that nested-key-null-strategy is only configured when nested-key is present. With a default value, these two cases become indistinguishable. The default behavior is still preserved by falling back to MERGE in FieldNestedUpdateAgg.
I'm happy to adjust this if you think consistency with other options is more important.
There was a problem hiding this comment.
Why not create a default value?
Thanks for the suggestion! I think using a default value also makes sense.
The only issue is that once MERGE becomes the default, FieldNestedUpdateAgg can no longer distinguish between an unspecified option and an explicitly configured MERGE in its constructor, so the dependency validation cannot be performed there anymore.
Instead, I'm thinking of moving the validation to FieldNestedUpdateAggFactory.create(), where the dependency can be validated before creating the aggregator by checking whether nested-key-null-strategy was explicitly configured. This still provides fail-fast validation during table creation and avoids allowing table options that have no effect.
If this approach sounds reasonable, I'll update the implementation accordingly.
boolean strategyConfigured =
options.toConfiguration()
.containsKey(...);
checkArgument(
!strategyConfigured || !nestedKey.isEmpty(),
...);Something along these lines.
Purpose
[Feature] Add
fields.{fieldName}.nested-key-null-strategyconfiguration fornested_updatefunction[Feature] Support nested-key null handling strategies in FieldNestedUpdateAgg operator to control behavior when nested-key does not satisfy primary key semantics
Expected behavior when nested-key contains null values or does not satisfy primary key semantics:
In the following examples,
nested-keyis defined ask0,k1.Behavior Definition
mergeMerge rows even if nested-key does not satisfy primary key semantics.
This is equivalent to not configuring
nested-key-null-strategy.ignoreIgnore rows whose nested-key does not satisfy primary key semantics.
errorThrow an exception when nested-key does not satisfy primary key semantics.
Tests
API and Format
No Changes.
Documentation
docs/docs/primary-key-table/merge-engine/aggregation.mdx