Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v3.0.0] Deprecate OpenSearch DSL format #3367

Open
wants to merge 21 commits into
base: main
Choose a base branch
from

Conversation

seankao-az
Copy link
Collaborator

@seankao-az seankao-az commented Feb 28, 2025

Description

Removes the OpenSearch DSL response format.

This PR focus only on deprecating the format, and will silent the affected legacy test cases. #3372 is created to track the task to decide which of the legacy tests are worth keeping and what to do with them.

Documentation website changes required:

  • response_formats.md
  • limitation.md
  • sql-ppl-api.md

possibly more

Related Issues

Resolves #3280

Check List

- [ ] New functionality includes testing.

  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.

- [ ] API changes companion pull request created.

  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Sean Kao <[email protected]>
Signed-off-by: Sean Kao <[email protected]>
Signed-off-by: Sean Kao <[email protected]>
@seankao-az seankao-az changed the title Deprecate OpenSearch DSL format [v3.0.0] Deprecate OpenSearch DSL format Feb 28, 2025
Signed-off-by: Sean Kao <[email protected]>
Signed-off-by: Sean Kao <[email protected]>
Signed-off-by: Sean Kao <[email protected]>
Signed-off-by: Sean Kao <[email protected]>
Signed-off-by: Sean Kao <[email protected]>
Signed-off-by: Sean Kao <[email protected]>
Signed-off-by: Sean Kao <[email protected]>
@@ -116,7 +126,7 @@ public static QueryAction create(Client client, QueryActionRequest request)
} else {
sqlExpr.accept(new TermFieldRewriter());
// migrate aggregation to query planner framework.
if (shouldMigrateToQueryPlan(sqlExpr, request.getFormat())) {
if (!bypassMigrateToQueryPlan && shouldMigrateToQueryPlan(sqlExpr)) {
Copy link
Collaborator Author

@seankao-az seankao-az Mar 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously some test cases use Format.JSON to bypass this check and get QueryAction for SELECT (either AggregationQueryAction or DefaultQueryAction). After removing Format.JSON, those test case wouldn't work without this flag.

@seankao-az seankao-az marked this pull request as ready for review March 3, 2025 19:41
Signed-off-by: Sean Kao <[email protected]>
@seankao-az seankao-az self-assigned this Mar 3, 2025
Comment on lines -197 to -199
if (format == Format.JSON) {
return false;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now there is no JSON format, I'm thinking should we ignore/fix the failed IT instead of adding a flag bypassMigrateToQueryPlan?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

main problem trying to solve with this flag is CheckScriptContents.getScriptFieldFromQuery needs access to a SearchRequestBuilder in QueryAction, which I think QueryPlanQueryAction doesn't expose. Will double check if there's another way

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think for the tests that used this method, it's a simple fix to change the query to not have any aggregation. Doing so will avoid migrating to QueryPlanner as well.

Those are UTs testing the string operators, and the GROUP BY and ORDER BY aren't used anyway, so it should be safe to remove them.

@seankao-az seankao-az requested a review from dai-chen March 4, 2025 21:43
return create(client, sql, false);
}

public static QueryAction create(Client client, String sql, boolean bypassMigrateToQueryPlan)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function only used in UT? if so, add a tag/java-doc on this function.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i removed this in latest commit. It was an attempt to fix some test cases, but it turns out changing the test cases is easier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Deprecated OpenSearch DSL format.
3 participants