Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature] Support streaming of buffer/bytea parameters #408

Open
vicary opened this issue Sep 18, 2022 · 0 comments
Open

[feature] Support streaming of buffer/bytea parameters #408

vicary opened this issue Sep 18, 2022 · 0 comments
Labels
help wanted Extra attention is needed

Comments

@vicary
Copy link

vicary commented Sep 18, 2022

My app manages deep learning models and it's metadata, the models are sized around 50MB - 100MB.

I am using CockroachDB for storage because it has no hard limit on their BYTES / BYTEA / BLOB columns, it is handy when the binary data is infrequently accessed. The binary data is for records and statistical study, they are usually written once and rarely read.

I know it is not the best practice out there, but I don't want to setup object storage just for them.

With the current version, when the buffer goes more than ~30MB V8 will start throwing heap allocation failures, I believe denoland/rusty_v8#427 is not enough to catch up with the sudden memory spike.

Does it make sense if the parameters can be transferred via ReadableStream and WriteableStream?

@bombillazo bombillazo added the help wanted Extra attention is needed label Feb 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants