Skip to content

fix Optimization: Use Lucene's TwoPhaseIterator for quick filtering t… #129893

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: 7.17
Choose a base branch
from

Conversation

csywxr
Copy link

@csywxr csywxr commented Jun 24, 2025

Using Lucene's Two Phase Query to submit and optimize the match query, the query has been tested and found to be problem free

Copy link

cla-checker-service bot commented Jun 24, 2025

❌ Author of the following commits did not sign a Contributor Agreement:
3601f39, 5a42a2e, 2fec87f

Please, read and sign the above mentioned agreement if you want to contribute to this project

@elasticsearchmachine elasticsearchmachine added v7.17.30 needs:triage Requires assignment of a team area label external-contributor Pull request authored by a developer outside the Elasticsearch team labels Jun 24, 2025
@csywxr
Copy link
Author

csywxr commented Jun 24, 2025

Using Lucene's Two Phase Query submission to optimize match queries

1 similar comment
@csywxr
Copy link
Author

csywxr commented Jun 24, 2025

Using Lucene's Two Phase Query submission to optimize match queries

@PeteGillinElastic PeteGillinElastic added :Search Foundations/Search Catch all for Search Foundations and removed needs:triage Requires assignment of a team area label labels Jul 11, 2025
@elasticsearchmachine elasticsearchmachine added the Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch label Jul 11, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-foundations (Team:Search Foundations)

@javanna
Copy link
Member

javanna commented Jul 14, 2025

Heya @csywxr , thanks for opening this PR!

Could you please share more context on what originated this, what problems it solves, and also why it was opened against the 7.17 branch instead of main?

Thanks!

@csywxr
Copy link
Author

csywxr commented Jul 14, 2025

Hello Javanna
The 7.17 version is commonly used in our project, but the query efficiency is not as high as expected. Through analyzing the source code of Elasticsearch in 7.17, we found that we can use Lucense's TwoPhaseIterator to improve query efficiency

@csywxr
Copy link
Author

csywxr commented Jul 14, 2025

Heya @csywxr , thanks for opening this PR!

Could you please share more context on what originated this, what problems it solves, and also why it was opened against the 7.17 branch instead of main?

Thanks!

Hello Javanna
The 7.17 version is commonly used in our project, but the query efficiency is not as high as expected. Through analyzing the source code of Elasticsearch in 7.17, we found that we can use Lucense's TwoPhaseIterator to improve query efficiency

@javanna
Copy link
Member

javanna commented Jul 15, 2025

Thanks for getting back to me. Do you have benchmarks results of the impact? Asking because we have no plans to do any further 7.17 releases, bearing any catastrophic bugs found, and this does not seem like it's the case.

We would be open to making the change against main, and perhaps backport it to 8.19. Yet we need to work on adding tests and better understanding the usecase and gain.

Correct me if I am wrong but I care to specify that this isn't a change to the match query of the query DSL, but to the named queries feature.

Thanks again!

@csywxr
Copy link
Author

csywxr commented Jul 18, 2025

Thanks for getting back to me. Do you have benchmarks results of the impact? Asking because we have no plans to do any further 7.17 releases, bearing any catastrophic bugs found, and this does not seem like it's the case.

We would be open to making the change against main, and perhaps backport it to 8.19. Yet we need to work on adding tests and better understanding the usecase and gain.

Correct me if I am wrong but I care to specify that this isn't a change to the match query of the query DSL, but to the named queries feature.

Thanks again!

I aggree making the change against main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
external-contributor Pull request authored by a developer outside the Elasticsearch team :Search Foundations/Search Catch all for Search Foundations Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch v7.17.30
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants