[python] Skip empty batches when advancing merge readers

[wangzhigang1999]

Purpose

Fixes #9819.

Fix AttributeError: 'NoneType' object has no attribute 'key' in primary-key merge reads when an input reader returns an empty batch after a non-empty batch.

Change Element.update() to skip empty batches until it finds a record or reaches EOF. Return True only when a record is available, and close the reader at EOF.

Tests

Add one regression test with two sorted inputs, including consecutive empty batches between records and before EOF. Assert merged output, deduplication, and reader closure. The test fails with the reported exception on the unchanged baseline and passes with this fix.

Local validation after rebasing onto 43e9ad701ebee6d9af8316f35533f4d0836a1fc0, with Python 3.12.6 and PyArrow 19.0.1:

• Real local-file reproduction: full scan remains at 6,144 rows; filtered scan returns the expected four rows and values.
• All 59 selected tests passed: the new regression and existing sequence-field, primary-key reader, predicate reader, and merge-engine dispatch tests.
• Changed-file flake8, license headers, and git diff --check passed.

The full CI matrix has not been run locally.

[Copilot]

🟢 Approval recommended

No unresolved blocking issues were identified, and regression coverage is included.

Pull request overview

Fixes Python primary-key merge reads by skipping empty batches until a record or EOF.

Changes:

• Handles consecutive empty batches and closes readers at EOF.
• Adds regression coverage for ordering, deduplication, and reader closure.

File summaries

File	Description
paimon-python/pypaimon/tests/test_sort_merge_reader.py	Adds regression coverage for empty-batch transitions.
paimon-python/pypaimon/read/reader/sort_merge_reader.py	Safely skips empty batches during merge advancement.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[JingsongLi]

Requirement fit: SUPPORTED. Implementation: CLEAN.

Reviewed b1c77cfdbc41. Skipping empty batches fixes a reachable primary-key merge-read failure without changing the key/sequence ordering. The loop now follows the same empty-batch/EOF contract used when initially filling the heap, and it closes exhausted readers.

Validation: 43 tests passed across the new sort-merge regression, sequence-field tests and primary-key reader tests, using PyArrow 19.0.1. Current head CI is green.

No actionable implementation regression found in this review.