Usage with prodigy's pdf.layout.fetch recipe exits uncleanly

[thatbudakguy]

Filing this here since https://github.com/explosion/prodigy-pdf doesn't have issues enabled.

Running the pdf.layout.fetch recipe seems to always exit with:

ℹ Creating preprocessed PDFs
✔ Saved fetched data to local file
assets/my_file.jsonl
-> Cannot close object, library is destroyed. This may cause a memory leak!
zsh: bus error python -m prodigy pdf.layout.fetch assets/my_file.jsonl

The output is there, but the memory leak warnings seem like a concern. I see similar output if I CTRL+C during docling parsing; maybe the prodigy recipe is causing the parser to exit prematurely?

You can reproduce this with this PDF by putting it in a directory called pdf_test and running something like:

python -m prodigy pdf.layout.fetch assets/test_output.jsonl blank:en pdf_test

Software versions:

• spaCy 3.7.5
• prodigy 1.17.2
• spacy_layout 0.0.9
• prodigy_pdf 0.4.0
• docling 2.13.0

Hardware: Apple M3 Pro, MacOS 14.7.1

[thatbudakguy]

(I can move this to the prodigy forum if desired; just realized that's why issues are not enabled there!)