Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

_ #22

Open
ilanbm opened this issue Dec 2, 2020 · 1 comment
Open

_ #22

ilanbm opened this issue Dec 2, 2020 · 1 comment

Comments

@ilanbm
Copy link

ilanbm commented Dec 2, 2020

No description provided.

@ilanbm ilanbm changed the title Can't upload CSV, getting 'Data cannot be cloned, out of memory.' _ Dec 2, 2020
@talolard
Copy link
Collaborator

talolard commented Dec 2, 2020

Thanks for giving this a try and sharing the issue.

Here's what's happening and what we can do about it:
When the user submits a file we pass at to a webworker via a singleton wrapper over the worker.
That happens here

The issue you hit on is that passing data to webworks requires cloning the data and you ran out of memory during the clone.
The solution for that is to modify the code above to batch the inserts.

It might be tempting to just do the insert from the main thread but

  1. That would block the UI
  2. Indexddb (where we store the data) has a limit on how much data we can send to it in one chunk so we'd need to batch anyway

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants