Repository navigation
Conversation
Every fill whose colour is a tiling pattern replays the pattern cell on a lattice of up to 17x17 nested stream decoders. Without shape tracking the path being filled is never recorded, so the lattice falls back to the page box and runs in full for every fill, and every stroke it produces is then dropped by the same shape guards. A CAD sheet hatched with thousands of pattern fills took minutes to decode for text alone. Return early from the tiling branch when `keep_shapes` is false, the same way the shading-pattern branch already does, and parse the pattern cell once per resource instead of once per fill. Fixes docling-project#375 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Jeff Witt <1848307+wittjeff@users.noreply.github.com>
… the cell A tiling-pattern fill replayed the pattern cell on a lattice of up to 17x17 nested stream decoders, registering every cell stroke as a page shape and a render instruction. A CAD sheet hatched with thousands of regions produced millions of them and took minutes to decode with shapes kept, which the threaded backend always does for its page render. The cell is now decoded once per pattern (and per colour for an uncoloured one) into its own instruction list, in pattern space. A fill emits one shape instruction for its path carrying that cell as a tiling paint. The Blend2D renderer rasterises the cell into a tile at canvas resolution, caches it per page, and fills the path with a repeating BLPattern through the usual clip, mask and blend handling. Cells wider than their step are drawn again shifted by a step so the overlap wraps into the tile. Cells larger than the page keep the direct replay. The page keeps one shape item per hatched region, its outline, instead of the replayed strokes. Text inside a pattern cell is tile content and is no longer reported as page text. Refs docling-project#375 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Jeff Witt <1848307+wittjeff@users.noreply.github.com>
|
✅ DCO Check Passed Thanks @wittjeff, all your commits are properly signed off. 🎉 |
Merge Protections🟢 Merge protection satisfied — ready to merge. Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
…undtruth Dataset PR: https://huggingface.co/datasets/docling-project/regression-dataset-for-docling-parse/discussions/8 Re-point HF_DATASET_REVISION at the merge commit once that PR lands. Signed-off-by: Jeff Witt <1848307+wittjeff@users.noreply.github.com>
Regression suite
Two corrections to the table: Regenerated groundtruth: dataset PR https://huggingface.co/datasets/docling-project/regression-dataset-for-docling-parse/discussions/8 (head c958820d) (186 files: 44 parser |
Stacked on #386 (first commit); this PR is the second commit. Follow-up to #375 for the shapes-COMPUTE path, as laid out in this comment.
Problem
A tiling-pattern fill replayed the pattern cell on a lattice of up to 17x17 nested stream decoders, registering every cell stroke as a page shape and a render instruction. With shapes kept, which the threaded backend always does for its page render, a CAD sheet hatched with thousands of regions produced millions of them. Measured with the 7.22.2 wheel through
DoclingThreadedPdfParser(shapes COMPUTE, one thread, 1684x2384 pt page, 500 hatched 60x60 pt regions, 4 pt cell): 12.7 s to decode and 16.4 s with the render for a one-stroke cell, 171 s and 185 s for a 30-stroke cell. #386 does not reach this path.Changes
Decoder:
pdf_resource<PAGE_PATTERN>reads/PaintTypeand caches the decoded cell per paint colour key.do_pattern_filldecodes the cell once (make_tiling_paint): a freshpdf_decoder<STREAM>with its own cell, shape and image sinks and an identity CTM, so the result is in pattern space; for an uncoloured pattern the scn operand colour is set first. The cell is packaged with the pattern-space to page-space matrix, steps and bbox as atiling_paint, and handed to the shape state as the pending paint of the next painting operator.register_pathsattaches that paint to the fill instruction it emits (fill, or fill+stroke with the stroke unchanged) instead of skipping the fill as unresolved. The page keeps one shape item per fill, the outline.report_tiling_cell_images), so a textured fill still counts as image content for docling's OCR-region decisions without the lattice of placements.Renderer:
render_shapewith a tiling paint rasterises the cell into a tile at canvas resolution (rasterize_tile: a child renderer over one step box, transparent background, drawn again at the step offsets when the cell's bbox exceeds its step so the overlap wraps), caches it per page by cell and tile size within a 64 MB budget (tiles are capped at 2048 px a side and 4 Mpx), and fills the path with aBLPattern(tile, REPEAT, transform). Clip, clip mask, alpha and blend mode go through the existing code.pdf_render_instructions::iterate_over_contentdraws a list without its size instruction, optionally translated;bitmap_instruction::translatedadded for it.Measurements
Same page set, same machine, both builds under the same load:
Annex L of ISO 32000-1 (page 745 of the spec, figures L.8 and L.9, the coloured and uncoloured tiling examples) renders the same as before: mean absolute pixel difference 0.2 at scale 2, differences only on glyph edges.
Behaviour changes
intersects_with(shapes=True)still sees vector content there;get_shape_linesno longer sees hatch strokes as candidate rules.Tests
New in
tests/test_unit_patterns.py: one shape per hatched region with the region still painted at the cell's coverage; a page-sized cell still painted (replay fallback); a cell wider than its step wrapping into the tile; an image cell reported as one page image spanning the fill and still painted; a cell that paints with its own pattern terminating. The #386 test now asserts the cell's text is not page text and the fill is still painted. The existing pattern tests (placement, anchoring, both lattice directions, path bound, even-odd, uncoloured colour, fill+stroke, unresolved pattern) pass unchanged:tests/test_unit_patterns.py tests/test_unit_clipping.py tests/test_unit_jbig2_masks.py tests/test_unit_shadings.py: 47 passed.Regression suites
test_regression_threaded_render.py,test_regression_parse.pyandtest_regression_threaded_parse.pywere run on this branch and on its base (#386) against the same groundtruth, in parallel on the same machine.3763025099401294496-2.pdfpages 7-9 (line cells vs groundtruth) and the Type3 bitmap metadata oftext_as_lines_01.pdf.verify_SegmentedPdfPage(shapes and bitmap resources) rather than in the text comparisons except where noted:PDF32000_2008.pdfc9612bab-d235-4788-8194-6cc78b4d7456-1.pdf2508.13113v2.pdf3763025099401294496-2.pdf11033738705194660817-2.pdf11273518440839632455_002.pdf14570626672493314948-53.pdf14911938403731335361-1.pdf15424333971388669153-2.pdfThe groundtruth for those pages needs regenerating with the dataset; I cannot push the dataset from here. Everything else in the corpus is unchanged: a count diff of chars, lines, shapes and bitmaps over all 3369 pages between the two builds lists only the pages above.
Rendering of those pages, this branch against its base at scale 2: mean absolute pixel difference at most 0.96 (
11273518440839632455_002.pdfpage 4) and below 0.1 on 30 of the 42 pages. The visible differences are the hatched rules in the newsletter now running their full length, and tile-edge anti-aliasing on the vector hatches.🤖 Generated with Claude Code