feat: Iceberg V2 delete file support in druid-iceberg-extensions#19266

Open

Shekharrajak wants to merge 51 commits into

apache:masterfrom

Shekharrajak:feature/iceberg-v2-delete-support

Commits on May 27, 2026

add V2DeleteHandling enum for iceberg v2 delete file support
Shekharrajak
committed
add iceberg-data dependency and promote iceberg-parquet to compile scope
Shekharrajak
committed
implement IcebergRecordConverter for iceberg Record to Druid map conversion
Shekharrajak
committed
implement streaming IcebergNativeRecordReader with v2 delete support
Shekharrajak
committed
add FileScanResult and extractFileScanTasks to IcebergCatalog for v2 delete detection
Shekharrajak
committed
wire v2DeleteHandling into IcebergInputSource with native reader routing
Shekharrajak
committed
add unit tests for IcebergRecordConverter and V2DeleteHandling
Shekharrajak
committed
fix compilation: use public Iceberg APIs and correct Druid method signatures
Shekharrajak
committed
fix test compilation and all test failures for v2 delete handling
Shekharrajak
committed
cleanup temporary settings file
Shekharrajak
committed
remove V2DeleteHandling config and add auto-detect routing in IcebergInputSource
Shekharrajak
committed
add DeleteFileInfo POJO for serializable delete file metadata
Shekharrajak
committed
add IcebergFileTaskInputSource for per-task serializable v2 input source
Shekharrajak
committed
rewrite IcebergNativeRecordReader with manual positional and equality delete application
Shekharrajak
committed
add CompositeInputSourceReader for multi-task v2 reading in IcebergInputSource
Shekharrajak
committed
register IcebergFileTaskInputSource in IcebergDruidModule
Shekharrajak
committed
update all tests for auto-detect v2 architecture without V2DeleteHandling config
Shekharrajak
committed
fix missing Iterator import in IcebergInputSource
Shekharrajak
committed
docs: add iceberg v2 delete file support documentation
Shekharrajak
committed
fix checkstyle, strict compilation, and double-brace initialization errors
Shekharrajak
committed
add v2 table creation overload to IcebergRestCatalogResource
Shekharrajak
committed
add IcebergV2DeleteIngestionTest with equality, positional, mixed, and no-delete scenarios
Shekharrajak
committed
fix IT compilation: correct GenericParquetWriter method reference and verifySqlQuery signature
Shekharrajak
committed
fix dependency:analyze by scoping iceberg-data as runtime and adding to ignored list
Shekharrajak
committed
remove stray processing/.factorypath IDE artifact
Shekharrajak
committed
remove all stray .factorypath IDE artifacts from tracking
Shekharrajak
committed
Iceberg: reject unsupported delete file content types explicitly
Shekharrajak
committed
Iceberg: route v2 reader I/O through catalog FileIO instead of empty Hadoop config
Shekharrajak
committed
Iceberg: test positional delete file scoping across multiple data files
Shekharrajak
committed
Iceberg: test equality delete with NULL value matches NULL rows
Shekharrajak
committed
Iceberg: extract delete-file mapping into testable helper with rejection coverage
Shekharrajak
committed
Iceberg: test partitioned v2 table delete scoping
Shekharrajak
committed
Iceberg: make v2 path splittable so MSQ and parallel batch ingest rows
Shekharrajak
committed
Iceberg: make withSplit and getSplitHintSpecOrDefault self-loading
Shekharrajak
committed
Fix missing @Override on setup/tearDown in IcebergV2DeleteIngestionTest
Shekharrajak
committed
Call super.setup/tearDown in IcebergV2DeleteIngestionTest to start cluster
Shekharrajak
committed
Fix unclosed Iceberg writer before toDataFile/toDeleteFile calls
Shekharrajak
committed
Fix GenericParquetWriter::buildWriter -> ::create for Iceberg 1.10
Shekharrajak
committed
Guard non-Parquet Iceberg formats with UnsupportedOperationException (#19472)
Shekharrajak
committed
Fix MSQ ClassCastException, null StructLike, and ORDER BY issues in Iceberg V2 delete tests
Shekharrajak
committed
Fix positional delete writer: use withSpec instead of forTable to avoid row data requirement
Shekharrajak
committed
Guard delete file formats independently and document Parquet-only limitation (#19472)
Shekharrajak
committed
Fix DeleteFileInfo constructor calls in V2DeleteHandlingTest after adding format field
Shekharrajak
committed
fix(iceberg): promote iceberg-data to compile scope and drop stale unused-dep suppression
Shekharrajak
committed
fix(iceberg): plumb catalog Hadoop Configuration through to V2 native reader
Shekharrajak
committed

Commits on Jun 1, 2026

iceberg: exclude transitive orc-core from iceberg-data to avoid ORC classpath collision with druid-orc-extensions
Shekharrajak
committed