You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently we store the entire response body before returning it. While that works, certain datasets have very large response body. It would be inefficient to store it in memory, instead we can streams to a specified destination.
Proposed API Change(s)
Add support to stream the dataset response to a destination specified by the client. This can be done by adding streamTo as an option. The streamTo receives a stream destination.
Example saving the dataset content to a json file:
The example above demonstrates specifying a writable stream which then creates a file output.json filled with the response body
Breaking changes?
This change will require us to switch from using cross-fetch to node-fetch. Which means this package will no longer be isomorphic. Now do we only support node environment? Do we provide fallback if in a browser environment?
The text was updated successfully, but these errors were encountered:
@hawyar - This change makes sense to me. What would supporting both environments look like to you? Requiring both dependencies and then including in our documentation how to use client in browser vs. node? Or did you have something else in mind?
P.S. Once again, awesome documentation here! You're setting a great example for the team for effective communication. A+
The only difference between the APIs would be in node you can stream the response but in the browser you can't. We will provide documentation and the rest should remain the same. For the browser we can use esbuild to bundle cross-fetch with our package. I would love to hear other ways of handling this. Thank you @ccali11
Background
Currently we store the entire response body before returning it. While that works, certain datasets have very large response body. It would be inefficient to store it in memory, instead we can streams to a specified destination.
Proposed API Change(s)
Add support to stream the dataset response to a destination specified by the client. This can be done by adding
streamTo
as an option. ThestreamTo
receives a stream destination.Example saving the dataset content to a json file:
The example above demonstrates specifying a writable stream which then creates a file
output.json
filled with the response bodyBreaking changes?
This change will require us to switch from using cross-fetch to node-fetch. Which means this package will no longer be isomorphic. Now do we only support node environment? Do we provide fallback if in a browser environment?
The text was updated successfully, but these errors were encountered: