Why we don't use Pymudf or pdfium for data extraction ? OCR takes lot of time

[suryadev777]

for extracting data from a pdf why don't we use Pymudf or pdfium ? is there any drawback on them?

[cau-git]

@suryadev777 We are applying OCR only in cases where we detect a bitmap resource embedded in the PDF by default. This is to accomodate scanned PDFs.

For the bulk of programmatic PDFs, we do not use OCR. We implemented an own PDF parser (https://github.com/DS4SD/docling-parse) for text element extraction, which is an open alternative to pymupdf and pypdfium. Also, we offer a pypdfium-based PDF parser backend as an alternative choice in docling, see here.

[suryadev777]

Thank you for the response but i have a few question

1. what is the reason to pick pypdfium as a default pdf parser ?
2. After extraction the pdf object how we classify its a paragraph , heading ? is simply based on size threshold?
3. how we detect its a bitmap or regular text ?

[cau-git]

@suryadev777 I would advise you to read our technical report: https://arxiv.org/abs/2408.09869

To answer in short to your points:

1. pypdfium is not our default parser, it is docling-parse. We offer pypdfium as an alternative choice.
2. After extraction, we run a full pipeline of AI models for the tasks of layout detection, table structure recognition, and other features.
3. The PDF backend (docling-parse or pypdfium) have APIs to return bitmap rectangles on a given PDF page.

[suryadev777]

Thank you