Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve file reading performance #18

Open
cristianlivella opened this issue Dec 7, 2021 · 1 comment
Open

Improve file reading performance #18

cristianlivella opened this issue Dec 7, 2021 · 1 comment
Labels
performance Performance related issue

Comments

@cristianlivella
Copy link
Owner

cristianlivella commented Dec 7, 2021 •
edited
Loading

PhpSpreadsheet uses a lot of memory in order to read the input file.

Roughly ~400 MB of RAM are used for elaborate a report with ~200.000 transactions. Of these, about 350 MB are used by PhpSpreadsheet.

This seems to be a known and frequent problem: https://phpspreadsheet.readthedocs.io/en/latest/topics/memory_saving.

We need to consider replacing it with Spout, because of the supported file formats (csv, xsls and ods).

Spout seems to use much less memory (they say 3 MB, regardless of file size), but seems to be less smart about recognizing file types and delimiters for CSV files.

We could use PhpSpreadsheet to recognize the file type and delimiters, and then Spout to actually read the file.
@cristianlivella cristianlivella added the performance Performance related issue label Dec 7, 2021
@el1jaC
Copy link

el1jaC commented Oct 10, 2022

Are you sure to choose Spout?
It seems to be an archive project, and in future could be a problem if a security issue will be discovered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
performance Performance related issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@el1jaC @cristianlivella