[fix](fe) Fix PullUpPredicates losing NULL rows in INTERSECT/EXCEPT#62299
Open
yujun777 wants to merge 1 commit intoapache:masterfrom
Open
[fix](fe) Fix PullUpPredicates losing NULL rows in INTERSECT/EXCEPT#62299yujun777 wants to merge 1 commit intoapache:masterfrom
yujun777 wants to merge 1 commit intoapache:masterfrom
Conversation
### What problem does this PR solve? Issue Number: close #xxx Problem Summary: PullUpPredicates.getFiltersFromUnionConstExprs() incorrectly removes NullLiteral from UNION ALL constant expressions without compensating with OR IS NULL, producing invalid pull-up predicates. For example, SELECT 1 UNION ALL SELECT 3 UNION ALL SELECT NULL generates the pull-up predicate n IN (1, 3), which is NOT true for the NULL row (NULL IN (1, 3) evaluates to NULL, not TRUE). When InferPredicates pushes this predicate from one INTERSECT child to another, the IN filter incorrectly eliminates NULL rows. Since INTERSECT treats NULL=NULL per SQL standard, this causes incorrect query results with missing NULLs. Fix: when NULL constants are present, generate n IN (...) OR n IS NULL instead of just n IN (...), making the predicate valid for all rows including NULL. ### Release note Fixed a bug where INTERSECT and EXCEPT queries could incorrectly drop NULL rows when the optimizer inferred and pushed down predicates from UNION ALL constant expressions. ### Check List (For Author) - Test: Regression test - Behavior changed: No - Does this need documentation: No Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Contributor
Author
|
/review |
Contributor
|
No issues found in this review. Critical checkpoints:
Overall opinion: acceptable as-is based on the reviewed diff and surrounding optimizer flow. |
Contributor
FE UT Coverage ReportIncrement line coverage |
morrySnow
approved these changes
Apr 10, 2026
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #xxx
Problem Summary:
introduced by #39450
PullUpPredicates.getFiltersFromUnionConstExprs()incorrectly removesNullLiteralfrom UNION ALL constant expressions without compensating withOR IS NULL, producing invalid pull-up predicates.For example,
SELECT 1 UNION ALL SELECT 3 UNION ALL SELECT NULLgenerates the pull-up predicaten IN (1, 3), which is NOT true for the NULL row (NULL IN (1, 3)evaluates to NULL, not TRUE).When
InferPredicatespushes this predicate from one INTERSECT child to another, the IN filter incorrectly eliminates NULL rows. Since INTERSECT treats NULL=NULL per SQL standard, this causes incorrect query results with missing NULLs.Root Cause: In
getFiltersFromUnionConstExprs(), line 467:removes NULL but does not add
OR IS NULLto compensate.Fix: When NULL constants are present, generate
n IN (...) OR n IS NULLinstead of justn IN (...), making the predicate valid for all rows including NULL.Reproducing Query:
Release note
Fixed a bug where INTERSECT and EXCEPT queries could incorrectly drop NULL rows when the optimizer inferred and pushed down predicates from UNION ALL constant expressions.
Check List (For Author)