Handling rich data files such as PDF/CSV/XLSX

[rjtormis]

Hi everyone!
I’d like to ask how you usually handle document ingestion for large files — for example, CSVs, XLSX, or PDFs.
I’m currently facing out-of-memory (OOM) issues when uploading rich data files. Our production server has 24 GB of GPU memory, but I’d like to make the process as optimized as possible.
Locally, I’m getting the following error on a Mac with a 16 GB GPU:

Error: MPS backend out of memory (MPS allocated: 14.31 GB, other allocations: 96.77 MB, max allowed: 18.13 GB). Tried to allocate 5.01 GB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).

Here is our current set up with chunking. For Embedding we are using Qwen-Embedding-0.6B.

        document_splitter = DocumentSplitter(
            split_by="sentence", split_length=8, split_overlap=2, split_threshold=4
        )
        csv_splitter = DocumentSplitter(
            split_by="line", split_length=100, split_overlap=10, split_threshold=20
        )
        xlsx_splitter = DocumentSplitter(
            split_by="line", split_length=100, split_overlap=10, split_threshold=20
        )

Any advice is appreciated

[sjrl]

Hey @rjtormis I believe your question was already answered in our discord here

Reproducing text for completeness

What's the batch size for embedder? I guess you could lower that to 4-5 and see? Your splitter looks good but still do for just one file and check if it is working as expected and breaking into proper chunks. But I guess batch size is the issue