Skip to content

[FLINK-37791][6/N] model sql to rel converter #26603

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jun 4, 2025

Conversation

lihaosky
Copy link
Contributor

@lihaosky lihaosky commented May 28, 2025

What is the purpose of the change

Update rel converter to handle SqlModelCall.

Brief change log

  • Update rel converter to handle SqlModelCall
  • Discover ModelProvider and wrap it inside RexModelCall for optimizer
  • Add rule to handle ml_predict call

Verifying this change

unit test

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (yes)
  • If yes, how is the feature documented? (JavaDocs)

@flinkbot
Copy link
Collaborator

flinkbot commented May 28, 2025

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Copy link
Member

@fsk119 fsk119 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution. I left some comments.

ModelProviderModel modelProviderModel, RelDataType inputType, RelDataType outputType) {
super(
outputType,
new SqlSpecialOperator("RexModelCall", SqlKind.OTHER_FUNCTION),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to introduce a new operator here? It's better we can reuse the SqlModelOperator.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just found SqlModelOperator needs CatalogSchemaModel as argument which we don't have inside ModelProviderModel

Copy link
Contributor Author

@lihaosky lihaosky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I messed up the commits somehow and squashed the commits and force pushed...

@fsk119
Copy link
Member

fsk119 commented May 29, 2025

It's ok. The change in the PR is not very large.

Copy link
Member

@fsk119 fsk119 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your update.

@fsk119
Copy link
Member

fsk119 commented May 29, 2025

I write my idea here. You can take a look 0.0

https://github.com/apache/flink/compare/master...fsk119:flink:converter2?expand=1

@lihaosky
Copy link
Contributor Author

I made most of the changes except reusing SqlModelOperator, I feel it's not clean to pass it along the stack and RexTableArgCall also use a new one: https://github.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/calcite/RexTableArgCall.java#L48

Copy link
Member

@fsk119 fsk119 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your update. I left some comments.

Copy link
Member

@fsk119 fsk119 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your update. I left some minor suggestions.

Copy link
Member

@fsk119 fsk119 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Merging when CI passes...

@fsk119 fsk119 merged commit 2df2286 into apache:master Jun 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants