-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HIVE-27102: Upgrade Calcite to 1.33.0 and Avatica to 1.23.0 #5196
Open
soumyakanti3578
wants to merge
83
commits into
apache:master
Choose a base branch
from
soumyakanti3578:calcite-upgrade-1.33
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
83 commits
Select commit
Hold shift + click to select a range
2745ddb
Bump Calcite version to 1.32.0
zabetak 9089775
Resolve compilation failure in hive-exec due to missing Value annotation
zabetak b408e1b
Compilation fails in JdbcHiveTableScan due to breaking changes in sup…
zabetak 3970dbd
Compilation error in HiveFilterJoinRule due to RelRule.Config.EMPTY
zabetak 3e37f3a
Revert "Compilation error in HiveFilterJoinRule due to RelRule.Config…
zabetak ef77e7e
Compilation error in HiveFilterJoinRule due to RelRule.Config.EMPTY
zabetak d742984
Compilation fails in JDBCExtractJoinFilterRule due to removed supercl…
zabetak 70d44e5
Compilation fails in HiveDruidRules due to RelRule.Config.EMPTY
zabetak 47e2cd9
Simplify HiveDruidRules by using the config from respective rules in …
zabetak af11032
Compilation fails in HiveRelFactories due to ProjectFactory signature…
zabetak c323369
NoSuchFieldError: BACK_TICK_BACKSLASH when running TestMiniLlapLocalC…
zabetak f3fb8a7
NoSuchMethodError: CompilerFactoryFactory.getDefaultCompilerFactory w…
zabetak 550b5d9
Compilation fails in HiveRemoveEmptySingleRules due to RelRule.Config…
zabetak 4bc872f
Compilation fails in HiveReduceExpressionsRule due to missing XxReduc…
zabetak 403e339
Bump Calcite version to 1.33.0 and Avatica to 1.23.0
zabetak 4aec55e
Compilation fails in HiveProjectFactoryImpl due to new method added i…
zabetak ca0ed83
StackOverflowError while running cbo_query13.q
zabetak f600a70
ClassCastException in RelMdSize#typeValueSize when compiling cbo_quer…
zabetak ef971a4
use RexSimplify in point look up rule
kasakrisz 6d64487
Correlate related prune empty rules config
kasakrisz 1c1e879
Iceberg MV related rules config
kasakrisz 1dc9066
Correlate related prune empty rules config
kasakrisz cc88810
transform search in ASTConverter
kasakrisz 1f55763
support cbo.returnpath
kasakrisz 3e2eadf
Add EPL-2.0 license to hive-packaging
soumyakanti3578 32f33d8
Expand SEARCH in HiveFilterSortPredicates to compute cost
soumyakanti3578 9b01d36
Extract RexInputRef in SEARCH Op when it is wrapped with a CAST
soumyakanti3578 59c4a58
Fix error caused by use of deprecated API which was removed
soumyakanti3578 99432b1
Error due to IN Operator in RexCall
soumyakanti3578 7527c54
Changes for cbo_query.* with TestTezTPCDS30TBPerfCliDriver
soumyakanti3578 e995603
clean up searchOperandToAST method to handle different visitors
soumyakanti3578 3f8c0fe
ClassCastException because of trying to convert every operand to RexI…
soumyakanti3578 882a6f6
Add HivePointLookupOptimizerRules to simplify some RexNodes which are…
soumyakanti3578 bfc7036
Delete HiveSearchExpandRule.java
soumyakanti3578 8719d9d
Fix AST generation for literals with struct types
soumyakanti3578 9650357
changes in cbo_query*.q.out
soumyakanti3578 6e3b461
Add jts-core to beeline to try to fix init-metastore test failures in…
soumyakanti3578 53a87bc
Fix error in null to complex type handling
soumyakanti3578 9673476
Convert Sarg with bounded range to BETWEEN
soumyakanti3578 1544101
revert some changes to q.outs
soumyakanti3578 a32343d
Fix testFilterIsRemovedAndVersionIntervalFromIsSetWhenFilterHasSnapsh…
soumyakanti3578 67ed666
Fix testFilterLeftIntactWhenItDoesNotHaveSnapshotIdPredicate
soumyakanti3578 cb62c1a
Fix TestMaterializedViewRebuild
soumyakanti3578 99e595b
Ensure bound types for lower and upper endpoints are CLOSED to check …
soumyakanti3578 03b9075
Convert Sarg with multiple ranges to BETWEEN and NOT BETWEEN in the A…
soumyakanti3578 6ec6fa3
Support SEARCH operator in FilterSelectivityEstimator
soumyakanti3578 b4ec480
benign changes
soumyakanti3578 e5c4750
Use HivePointLookupOptimizerRule to convert SEARCH to BETWEEN and NOT…
soumyakanti3578 cdfd39a
benign changes due to handling nulls in Sargs
soumyakanti3578 fc7546d
benign changes for TestTezTPCDS30TBPerfCliDriver
soumyakanti3578 1086b7f
Fix tests in TestHivePointLookupOptimizerRule
soumyakanti3578 72c8b3b
Allow cast for CHARs and VARCHARs to avoid getting cast to STRING
soumyakanti3578 701a800
benign changes to various qouts
soumyakanti3578 0177145
Implement visit(JdbcTableScan) which was removed in CALCITE-4640
soumyakanti3578 3ef61e2
Add external_jdbc_table2.q.out, although the change indicates that pr…
soumyakanti3578 2d3db32
Don't convert SEARCH to IN when pointCount is less than HIVE_POINT_LO…
soumyakanti3578 41ec5c9
Adding changes to various qouts
soumyakanti3578 d79cc8e
benign changes
soumyakanti3578 78da82e
Fix tests failing in TestFilterSelectivityEstimator
soumyakanti3578 b8e718c
Fix getRollup in SqlAggFunction
kasakrisz 1179d00
Plan changes after fixing rollup
soumyakanti3578 1ee5248
Remove unnecessary CASTs in ASTConverter
soumyakanti3578 d21e0f9
More benign changes
soumyakanti3578 4c0aa1c
Do not extract cast when its operand is a literal
soumyakanti3578 cc3c4e9
Changes in prepare_plan.q.out
soumyakanti3578 9a03aa9
changes to qouts
soumyakanti3578 6c69528
Clean up and resolve some smells
soumyakanti3578 f11f1e5
Support serialization and deserialization for SEARCH
soumyakanti3578 53f5a99
Add SORT_QUERY_RESULTS to pointlookup3.q to remove flakiness
soumyakanti3578 1753d22
add explain cbo plan to sharedwork.q
soumyakanti3578 7daac94
Exclude HivePreFilteringRule from cardinality_preserving_join_opt2.q
soumyakanti3578 5d30b23
Fix failure of TestSymlinkTextInputFormat#testCombine
soumyakanti3578 4a6bc18
benign changes for split0
soumyakanti3578 a067f61
Resolve review comments
soumyakanti3578 7bd8df7
benign changes after rebasing
soumyakanti3578 9b81180
Add Calcite version check for 1.35
soumyakanti3578 0ee9602
Refactor method that transforms ORs to INs and BETWEENs
soumyakanti3578 8f3f9d1
benign changes for correlationoptimizer8.q
soumyakanti3578 04c1530
Refactor method `makeNotBetween` to use the RexBuilder.makeBetween
soumyakanti3578 9724d26
Retry simplify until a RexNode is simplified completely, upto a max of 5
soumyakanti3578 47cf072
benign changes after rebasing
soumyakanti3578 ccd9904
Revert "Retry simplify until a RexNode is simplified completely, upto…
soumyakanti3578 9ec32be
Check if HivePreFilteringRule has been applied on a Filter based on i…
soumyakanti3578 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What causes the change in the number of rows and width ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like there should be 6 rows instead of 5.
Here's the insert statement,
insert into ice01 values (1, 'ABC'),(2, 'CBS'),(3, null),(4, 'POPI'),(5, 'AQWR'),(6, 'POIU'),(9, null),(8, 'POIKL'),(10, 'YUIO');
And the delete predicate is:
where id>4 OR id=2
. That gives us 6 rows with ids 2, 5, 6, 9, 8, 10.Also, I believe row counts are just estimated based on stats? I could be wrong though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, these row counts are estimates only. The TS has 7 rows. Not 9 nor 6.
The change is caused that one of the predicate in a Filter operator on top of the TS has changed. I highlight the change here, please check the CBO plans of this statement
1.25
1.33
In 1.33 an extra predicate is added:
<($0, 2)