Skip to content

feat(render): tile pattern fills in the renderer instead of replaying the cell - #390

Open
wittjeff wants to merge 3 commits into
docling-project:mainfrom
wittjeff:feat/tiling-pattern-render
Open

wittjeff wants to merge 3 commits into
docling-project:mainfrom
wittjeff:feat/tiling-pattern-render

Conversation

@wittjeff

@wittjeff wittjeff commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Stacked on #386 (first commit); this PR is the second commit. Follow-up to #375 for the shapes-COMPUTE path, as laid out in this comment.

Problem

A tiling-pattern fill replayed the pattern cell on a lattice of up to 17x17 nested stream decoders, registering every cell stroke as a page shape and a render instruction. With shapes kept, which the threaded backend always does for its page render, a CAD sheet hatched with thousands of regions produced millions of them. Measured with the 7.22.2 wheel through DoclingThreadedPdfParser (shapes COMPUTE, one thread, 1684x2384 pt page, 500 hatched 60x60 pt regions, 4 pt cell): 12.7 s to decode and 16.4 s with the render for a one-stroke cell, 171 s and 185 s for a 30-stroke cell. #386 does not reach this path.

Changes

Decoder:

  • pdf_resource<PAGE_PATTERN> reads /PaintType and caches the decoded cell per paint colour key.
  • do_pattern_fill decodes the cell once (make_tiling_paint): a fresh pdf_decoder<STREAM> with its own cell, shape and image sinks and an identity CTM, so the result is in pattern space; for an uncoloured pattern the scn operand colour is set first. The cell is packaged with the pattern-space to page-space matrix, steps and bbox as a tiling_paint, and handed to the shape state as the pending paint of the next painting operator.
  • register_paths attaches that paint to the fill instruction it emits (fill, or fill+stroke with the stroke unchanged) instead of skipping the fill as unresolved. The page keeps one shape item per fill, the outline.
  • A cell that draws an image is reported once more at page level as one image over the bbox of the filled path (report_tiling_cell_images), so a textured fill still counts as image content for docling's OCR-region decisions without the lattice of placements.
  • Cells larger than the page keep the direct replay (a handful of placements). Nested cell decodes and replays are capped at depth 4, so a cell that paints with its own pattern, directly or through a chain, is skipped with a warning instead of recursing.

Renderer:

  • render_shape with a tiling paint rasterises the cell into a tile at canvas resolution (rasterize_tile: a child renderer over one step box, transparent background, drawn again at the step offsets when the cell's bbox exceeds its step so the overlap wraps), caches it per page by cell and tile size within a 64 MB budget (tiles are capped at 2048 px a side and 4 Mpx), and fills the path with a BLPattern(tile, REPEAT, transform). Clip, clip mask, alpha and blend mode go through the existing code.
  • pdf_render_instructions::iterate_over_content draws a list without its size instruction, optionally translated; bitmap_instruction::translated added for it.

Measurements

Same page set, same machine, both builds under the same load:

page content wheel 7.22.2 decode / render this PR decode / render
500 hatch fills, 1-stroke cell 12.7 s / 16.4 s 0.15 s / 0.46 s
500 hatch fills, 30-stroke cell 171 s / 185 s 0.03 s / 1.4 s
2000 hatch fills 3.6 s (text-only decode) 0.13 s / 1.3 s
control: 500 clipped strokes 0.17 s / 0.9 s 0.20 s / 1.2 s

Annex L of ISO 32000-1 (page 745 of the spec, figures L.8 and L.9, the coloured and uncoloured tiling examples) renders the same as before: mean absolute pixel difference 0.2 at scale 2, differences only on glyph edges.

Behaviour changes

  • A pattern fill is one shape item (its outline) rather than its replayed cell strokes. docling's intersects_with(shapes=True) still sees vector content there; get_shape_lines no longer sees hatch strokes as candidate rules.
  • Text drawn inside a pattern cell is tile content and is no longer reported as page text. On the Annex L page that removes 688 ZapfDingbats suit-glyph cells that the replay had emitted, once per lattice position.
  • An image drawn inside a pattern cell is reported as one page image over the filled region instead of once per lattice placement (377 placements on one newsletter page became 8 images, one per hatched rule).

Tests

New in tests/test_unit_patterns.py: one shape per hatched region with the region still painted at the cell's coverage; a page-sized cell still painted (replay fallback); a cell wider than its step wrapping into the tile; an image cell reported as one page image spanning the fill and still painted; a cell that paints with its own pattern terminating. The #386 test now asserts the cell's text is not page text and the fill is still painted. The existing pattern tests (placement, anchoring, both lattice directions, path bound, even-odd, uncoloured colour, fill+stroke, unresolved pattern) pass unchanged: tests/test_unit_patterns.py tests/test_unit_clipping.py tests/test_unit_jbig2_masks.py tests/test_unit_shadings.py: 47 passed.

Regression suites

test_regression_threaded_render.py, test_regression_parse.py and test_regression_threaded_parse.py were run on this branch and on its base (#386) against the same groundtruth, in parallel on the same machine.

  • Image metrics: identical on every page both runs compared (the per-page table is the same to four decimals). The pages that use tiling patterns are compared directly below.
  • Pre-existing failures, the same on both: 3763025099401294496-2.pdf pages 7-9 (line cells vs groundtruth) and the Type3 bitmap metadata of text_as_lines_01.pdf.
  • New on this branch, all on pages that use tiling patterns, and all in verify_SegmentedPdfPage (shapes and bitmap resources) rather than in the text comparisons except where noted:
document pages what changed
PDF32000_2008.pdf 745, 752 Annex L pattern figures: shapes 18633 -> 39 on 752; on 745 the 688 ZapfDingbats cells drawn inside the cells are no longer page text
c9612bab-d235-4788-8194-6cc78b4d7456-1.pdf 1 vector hatches: shapes 60447 -> 16945
2508.13113v2.pdf 19 vector hatch: shapes 1242 -> 290 (within tolerance, not a failure)
3763025099401294496-2.pdf 2-6, 10-15 vector hatches: shapes 959 -> 394 per page, text identical
11033738705194660817-2.pdf 2-18 image-cell background pattern: 4-5 replayed bitmaps -> 1 page image over the fill per page
11273518440839632455_002.pdf 1, 3, 4 image-cell hatches: 377 replayed bitmaps -> 8 page images on page 1; the hatched rule under the heading is now hatched along its whole length, the replay had stopped at its 17-cell cap
14570626672493314948-53.pdf 34, 35, 37-39 image-cell hatches: 544 -> 16 bitmaps on page 34
14911938403731335361-1.pdf 1, 2 image-cell patterns: 153 -> 1 and 311 -> 7 bitmaps
15424333971388669153-2.pdf 3 12 of 2295 bitmaps are now page images over their fills

The groundtruth for those pages needs regenerating with the dataset; I cannot push the dataset from here. Everything else in the corpus is unchanged: a count diff of chars, lines, shapes and bitmaps over all 3369 pages between the two builds lists only the pages above.

Rendering of those pages, this branch against its base at scale 2: mean absolute pixel difference at most 0.96 (11273518440839632455_002.pdf page 4) and below 0.1 on 30 of the 42 pages. The visible differences are the hatched rules in the newsletter now running their full length, and tile-edge anti-aliasing on the vector hatches.

🤖 Generated with Claude Code

wittjeff and others added 2 commits October 6, 2026 11:20
Every fill whose colour is a tiling pattern replays the pattern cell on
a lattice of up to 17x17 nested stream decoders. Without shape tracking
the path being filled is never recorded, so the lattice falls back to
the page box and runs in full for every fill, and every stroke it
produces is then dropped by the same shape guards. A CAD sheet hatched
with thousands of pattern fills took minutes to decode for text alone.

Return early from the tiling branch when `keep_shapes` is false, the
same way the shading-pattern branch already does, and parse the pattern
cell once per resource instead of once per fill.

Fixes docling-project#375

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Jeff Witt <1848307+wittjeff@users.noreply.github.com>
… the cell

A tiling-pattern fill replayed the pattern cell on a lattice of up to
17x17 nested stream decoders, registering every cell stroke as a page
shape and a render instruction. A CAD sheet hatched with thousands of
regions produced millions of them and took minutes to decode with
shapes kept, which the threaded backend always does for its page render.

The cell is now decoded once per pattern (and per colour for an
uncoloured one) into its own instruction list, in pattern space. A fill
emits one shape instruction for its path carrying that cell as a tiling
paint. The Blend2D renderer rasterises the cell into a tile at canvas
resolution, caches it per page, and fills the path with a repeating
BLPattern through the usual clip, mask and blend handling. Cells wider
than their step are drawn again shifted by a step so the overlap wraps
into the tile. Cells larger than the page keep the direct replay.

The page keeps one shape item per hatched region, its outline, instead
of the replayed strokes. Text inside a pattern cell is tile content and
is no longer reported as page text.

Refs docling-project#375

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Jeff Witt <1848307+wittjeff@users.noreply.github.com>
@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

✅ DCO Check Passed

Thanks @wittjeff, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@PeterStaar-IBM
PeterStaar-IBM self-requested a review October 7, 2026 05:37
…undtruth

Dataset PR: https://huggingface.co/datasets/docling-project/regression-dataset-for-docling-parse/discussions/8
Re-point HF_DATASET_REVISION at the merge commit once that PR lands.

Signed-off-by: Jeff Witt <1848307+wittjeff@users.noreply.github.com>
@wittjeff

wittjeff commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Regression suite

regression-tests / run-regression fails on 44 pages, all of them pages that use tiling patterns and all listed in the table above. The [char]/[word]/[line] labels in the failure list are the cell unit the bitmap check happened to run under; there is one cause per page:

  • 41 pages: bitmap resource count mismatch (the replayed cell images became one page image per fill);
  • c9612bab-…-1.pdf p1 and PDF32000_2008.pdf p752: shape counts (60447 -> 16945, 18633 -> 39);
  • PDF32000_2008.pdf p745: the 688 ZapfDingbats cells drawn inside the Annex L pattern cell;
  • render: instruction count on c9612bab-…-1.pdf p1 (the first page compared, 50818 -> 7473).

Two corrections to the table: 3763025099401294496-2.pdf pages 2-15 use an image cell as well (255 replayed bitmaps -> 35 page images per page, shapes 959 -> 394), and its pages 7-9 are not pre-existing failures: CI passes them on the base; my local copy of the dataset was stale for those three pages.

Regenerated groundtruth: dataset PR https://huggingface.co/datasets/docling-project/regression-dataset-for-docling-parse/discussions/8 (head c958820d) (186 files: 44 parser .py.json.gz, the three text exports of p745, 44 render instruction/bitmap/page artifacts, 7 retained bitmaps; nothing added or removed). Text exports are byte-identical on the other 43 pages. With that revision pinned, test_regression_parse.py, test_regression_threaded_parse.py and test_regression_threaded_render.py pass on the seven documents (4 passed); the rest of the corpus is untouched. 34 files under groundtruth/render/bitmap_data (the replayed cell images, <doc>.page_no_<n>.bitmap_1.*) are no longer referenced by any bitmaps.json; I left them in place.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant