feat(writer): add PositionDeleteInput builder by laskoviymishka · Pull Request #3230 · apache/iceberg-rust

laskoviymishka · 2026-09-15T15:23:11Z

What changes are included in this PR?

Add PositionDeleteInputBuilder, a small helper that assembles a spec-conforming position delete RecordBatch from (path, pos) rows.

The batch has exactly the two required position delete columns (file_path: Utf8, pos: Int64), both non-null and carrying the reserved field ids, so it passes PositionDeleteFileWriter's per-batch validation and can be handed straight to that writer. An empty builder still produces a valid 0-row batch.

To keep a single source of truth for the field-id wiring, the Arrow schema is no longer hand-wired in the builder. This commit un-gates the previously test-only POSITION_DELETE_ARROW_SCHEMA static and position_delete_arrow_schema() helper in position_delete_writer.rs, making the helper pub(crate), and the builder now obtains the shared Arrow projection through it as a cheap Arc clone.

extend accepts any iterator of (Into, i64) rows, and build debug-asserts the paths/positions length invariant. The module docs note that rows are materialized into two heap Vecs before the batch is built (so very large inputs should use one builder per write batch), that the optional third row column is not supported yet, and that position delete files are a v2 construct which v3 forbids, so a format-version gate must be applied at the transaction/commit layer.

Which issue does this PR close?

Relates to #340 and #2218.

Are these changes tested?

yes, new and existing tests

Add PositionDeleteInputBuilder, a small helper that assembles a spec-conforming position delete RecordBatch from (path, pos) rows. The batch has exactly the two required position delete columns (file_path: Utf8, pos: Int64), both non-null and carrying the reserved field ids, so it passes PositionDeleteFileWriter's per-batch validation and can be handed straight to that writer. An empty builder still produces a valid 0-row batch. To keep a single source of truth for the field-id wiring, the Arrow schema is no longer hand-wired in the builder. This commit un-gates the previously test-only POSITION_DELETE_ARROW_SCHEMA static and position_delete_arrow_schema() helper in position_delete_writer.rs, making the helper pub(crate), and the builder now obtains the shared Arrow projection through it as a cheap Arc clone. extend accepts any iterator of (Into<String>, i64) rows, and build debug-asserts the paths/positions length invariant. The module docs note that rows are materialized into two heap Vecs before the batch is built (so very large inputs should use one builder per write batch), that the optional third row column is not supported yet, and that position delete files are a v2 construct which v3 forbids, so a format-version gate must be applied at the transaction/commit layer. Relates to apache#340 and apache#2218.

…osition_delete_batch fn Per review, collapse the PositionDeleteInputBuilder surface (new/with_capacity/push/extend/len/is_empty/build) into a single free function, position_delete_batch(rows). A FooBuilder is expected to build a Foo, but the builder's build() only returned a bare RecordBatch, so a plain function reads more clearly and keeps the API as tight as possible. The function collects rows into a BTreeMap<String, RoaringTreemap> keyed by file path. This deduplicates repeated (file_path, pos) pairs and, when iterated, yields the spec-required ascending (file_path, pos) order for free, so callers no longer have to pre-sort their input. A negative pos is rejected with DataInvalid. The batch is still built from the writer's shared crate internal Arrow schema, and empty input yields a valid 0-row batch.

…Deletes accumulator Following blackmwk's review of the redesigned position delete assembly, replace the free position_delete_batch function with a small crate-internal PositionDeletes accumulator. Callers build up deletes with insert(path, pos) and render them once via to_record_batch(). Positions are now taken as u64: a row position is non-negative by construction, so the old negative-i64 rejection is gone. The batch's Int64 pos column is filled with a checked u64-to-i64 conversion that returns DataInvalid rather than panicking on a value past i64::MAX. Rows stay keyed by a BTreeMap<String, RoaringTreemap>, so iteration yields the spec-required (file_path, pos) order with duplicate pairs removed, and the two columns are now built with Arrow StringBuilder and Int64Builder instead of intermediate Vecs. The batch is assembled against the writer's shared position_delete_arrow_schema, and an empty accumulator still yields a valid 0-row batch. Nothing outside the crate consumes this yet, so the whole surface is pub(crate) and the module carries a dead-code allowance until the write-path plumbing lands. The public API drops the position_delete_batch entry accordingly.

laskoviymishka marked this pull request as ready for review September 15, 2026 20:02

anoopj reviewed Sep 16, 2026

View reviewed changes