Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Framework of Calcite Engine: Parser, Catalog Binding and Plan Converter #3249

Merged

Conversation

LantaoJin
Copy link
Member

@LantaoJin LantaoJin commented Jan 16, 2025

Description

Build the development framework of Calcite Engine which includes

Parser: Parse existing PPL ANTLR4 grammar to AST
Catalog Binding: Resolve and binding the OpenSearch Indics metadata as schema
Plan Converter: Convert the AST to Calcite logical plan (RelNode)

Related Issues

Resolves #3250

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@LantaoJin LantaoJin added the dev-branch PR is to merge into a dev branch label Jan 16, 2025
@LantaoJin
Copy link
Member Author

LantaoJin commented Jan 16, 2025

The IT failed during launching OpenSearch cluster with

[2025-01-16T18:36:12,516][ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [integTest-0] fatal error in thread [opensearch[integTest-0][sql-worker][T#1]], exiting
java.lang.ExceptionInInitializerError: null
	at org.apache.calcite.sql2rel.ReflectiveConvertletTable.registerNodeTypeMethod(ReflectiveConvertletTable.java:71) ~[?:?]
	at org.apache.calcite.sql2rel.ReflectiveConvertletTable.<init>(ReflectiveConvertletTable.java:55) ~[?:?]
	at org.apache.calcite.sql2rel.StandardConvertletTable.<init>(StandardConvertletTable.java:121) ~[?:?]
	at org.apache.calcite.sql2rel.StandardConvertletTable.<clinit>(StandardConvertletTable.java:115) ~[?:?]
	at org.apache.calcite.tools.Frameworks$ConfigBuilder.<init>(Frameworks.java:246) ~[?:?]
	at org.apache.calcite.tools.Frameworks$ConfigBuilder.<init>(Frameworks.java:227) ~[?:?]
	at org.apache.calcite.tools.Frameworks.newConfigBuilder(Frameworks.java:211) ~[?:?]
	at org.opensearch.sql.executor.QueryService.buildFrameworkConfig(QueryService.java:142) ~[?:?]
	at org.opensearch.sql.executor.QueryService.execute(QueryService.java:63) ~[?:?]
	at org.opensearch.sql.executor.execution.QueryPlan.execute(QueryPlan.java:65) ~[?:?]
	at org.opensearch.sql.opensearch.executor.OpenSearchQueryManager.lambda$submit$0(OpenSearchQueryManager.java:31) ~[?:?]
	at org.opensearch.sql.opensearch.executor.OpenSearchQueryManager.lambda$withCurrentContext$1(OpenSearchQueryManager.java:45) ~[?:?]
	at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:923) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
	at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessDeclaredMembers")
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:488) ~[?:?]
	at java.base/java.security.AccessController.checkPermission(AccessController.java:1071) ~[?:?]
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:411) ~[?:?]
	at java.base/java.lang.Class.checkMemberAccess(Class.java:3227) ~[?:?]
	at java.base/java.lang.Class.getDeclaredMethods(Class.java:2674) ~[?:?]
	at org.apache.calcite.util.ReflectUtil.<clinit>(ReflectUtil.java:63) ~[?:?]
	... 16 more

Even I have set systemProperty 'java.security.manager', 'disallow' or custom a policy file with systemProperty 'java.security.policy', "$policyFile"
the policy file includs

 grant {
    permission java.lang.RuntimePermission "accessDeclaredMembers";
 };

These IT failure could be reproduced in local. Check the log integ-test/build/testclusters/integTest-0/logs/opensearch.stdout.log

@LantaoJin LantaoJin marked this pull request as ready for review January 17, 2025 05:57
@LantaoJin LantaoJin merged commit 4b93d60 into opensearch-project:feature/calcite-engine Jan 17, 2025
6 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dev-branch PR is to merge into a dev branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant