Skip to content

Merge open_exchange with data transport call #128494

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

idegtiarenko
Copy link
Contributor

image

According to above typical request flow is:

  • open exchange on the data node (internal:data/read/esql/open_exchange)
  • start fetching data from it (internal:data/read/esql/exchange)
  • trigger computing data on the data node (indices:data/read/esql/data)

ElasticSearch relies on Netty to make transport as cheap and as performant as possible, but still each transport message has some cost associated with it.

This change:

  • merges open_exchange with data call to minimize amount of remote calls required to start a query on the data node
  • ensures internal:data/read/esql/exchange calls are started after above initialization is done to avoid making calls and receiving empty messages before actual data node computation starts

@idegtiarenko idegtiarenko requested review from nik9000 and dnhatn May 27, 2025 07:04
@idegtiarenko idegtiarenko added >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) :Analytics/ES|QL AKA ESQL v9.1.0 labels May 27, 2025
@dnhatn
Copy link
Member

dnhatn commented May 27, 2025

We started with a single call, but found handling out-of-order exchange requests too complex, so we decided to separate them.

@dnhatn
Copy link
Member

dnhatn commented May 27, 2025

This commit: 188b0a78

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v9.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants