Core: ManifestFilterManager thread safety issue during parallel overwrite commit

[liucao-dd]

Apache Iceberg version

1.10.2

Also appears present in 1.10.0, 1.10.1, 1.11.0, and main based on the current ManifestFilterManager implementation.

Query engine

Spark

Please describe the bug 🐞

A Spark OverwriteByExpression / full-table overwrite can crash during the Iceberg commit phase because ManifestFilterManager mutates a shared, non-thread-safe deleteFiles set from multiple manifest-filtering worker threads.

The crash observed was:

java.lang.ClassCastException: class java.util.LinkedHashMap$Entry cannot be cast to
class java.util.HashMap$TreeNode
    at java.base/java.util.HashMap$TreeNode.moveRootToFront(HashMap.java:1986)
    at java.base/java.util.HashMap$TreeNode.putTreeVal(HashMap.java:2165)
    at java.base/java.util.HashMap.putVal(HashMap.java:636)
    at java.base/java.util.HashMap.put(HashMap.java:610)
    at java.base/java.util.HashSet.add(HashSet.java:221)
    at org.apache.iceberg.util.WrapperSet.add(WrapperSet.java:97)
    at org.apache.iceberg.util.DataFileSet.add(DataFileSet.java:26)
    at org.apache.iceberg.ManifestFilterManager.lambda$filterManifestWithDeletedFiles$3(ManifestFilterManager.java:518)
    at java.base/java.lang.Iterable.forEach(Iterable.java:75)
    at org.apache.iceberg.ManifestFilterManager.filterManifestWithDeletedFiles(ManifestFilterManager.java:490)
    at org.apache.iceberg.ManifestFilterManager.filterManifest(ManifestFilterManager.java:370)
    at org.apache.iceberg.ManifestFilterManager.lambda$filterManifests$0(ManifestFilterManager.java:224)
    at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413)
    ...
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)

ManifestFilterManager.filterManifests processes manifests in parallel using the worker pool. For manifests that contain deleted files, filterManifestWithDeletedFiles adds copied files into the instance-level deleteFiles set:

deleteFiles.add(fileCopy);

That set is a DataFileSet, backed by WrapperSet, which uses a plain LinkedHashSet. Concurrent add calls can corrupt the underlying HashMap, which then surfaces as the LinkedHashMap$Entry cannot be cast to HashMap$TreeNode exception during bucket treeification.

This is especially likely for full-table overwrites because the delete expression is effectively alwaysTrue(): every live data file in every matching manifest is added to the same shared set while manifests are being filtered in parallel.

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time