Skip to content

[ENHANCEMENT] Streaming responses from Elasticsearch. #246

Open
@pvichivanives

Description

@pvichivanives

Is your feature request related to a problem? Please describe.
The feature request comes from trying to process many documents with a limited amount of ram. We would like to be able to access the data as a stream.

Describe the solution you'd like
Just an impl for response add for streaming. Something simple like below should work:

    /// Creates a bytes_stream from the response body.
    ///
    /// Reading the response body consumes `self`
    pub async fn bytes_stream(self) -> impl Stream<Item = Result<Bytes, ClientError>> {
        self.response.bytes_stream().map_err(ClientError::from)
    }

This could be put under the feature stream so that the library feature/imports are only included when people want to stream. This would allow the library to remain lightweight for people who don't want the future.
Describe alternatives you've considered
Alternatively elasticsearch could expose the underlying reqwest::Response which would allow users to stream from the underlying Response. This would however tie elasticsearch public API to include reqwest::Response.

Additional context
Happy to make a PR for this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions