Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement]: Equality field id are different in a RewriteFilesInput #2870

Open
2 of 3 tasks
XBaith opened this issue May 28, 2024 · 1 comment
Open
2 of 3 tasks

Comments

@XBaith
Copy link
Contributor

XBaith commented May 28, 2024

Search before asking

  • I have searched in the issues and found no similar issues.

What would you like to be improved?

tm_id=optimizer-kubed-bts-0-fo26ux-taskmanager-1-2
application_id=/default
java.lang.IllegalArgumentException: Equality delete files have different delete fields
	at org.apache.iceberg.relocated.com.google.common.base.Preconditions.checkArgument(Preconditions.java:143)
	at com.netease.arctic.io.reader.CombinedDeleteFilter.<init>(CombinedDeleteFilter.java:130)
	at com.netease.arctic.io.reader.GenericCombinedIcebergDataReader$GenericDeleteFilter.<init>(GenericCombinedIcebergDataReader.java:305)
	at com.netease.arctic.io.reader.GenericCombinedIcebergDataReader.<init>(GenericCombinedIcebergDataReader.java:97)
	at com.netease.arctic.optimizing.IcebergRewriteExecutor.dataReader(IcebergRewriteExecutor.java:68)
	at com.netease.arctic.optimizing.AbstractRewriteFilesExecutor.<init>(AbstractRewriteFilesExecutor.java:84)
	at com.netease.arctic.optimizing.IcebergRewriteExecutor.<init>(IcebergRewriteExecutor.java:46)
	at com.netease.arctic.optimizing.IcebergRewriteExecutorFactory.createExecutor(IcebergRewriteExecutorFactory.java:38)
	at com.netease.arctic.optimizing.IcebergRewriteExecutorFactory.createExecutor(IcebergRewriteExecutorFactory.java:25)
	at com.netease.arctic.optimizer.common.OptimizerExecutor.executeTask(OptimizerExecutor.java:148)
	at com.netease.arctic.optimizer.flink.FlinkOptimizerExecutor.executeTask(FlinkOptimizerExecutor.java:70)
	at com.netease.arctic.optimizer.common.OptimizerExecutor.start(OptimizerExecutor.java:52)
	at com.netease.arctic.optimizer.flink.FlinkExecutor.lambda$open$0(FlinkExecutor.java:59)
	at java.lang.Thread.run(Thread.java:750)

The process that have different equality field ids cannot be executed

How should we improve?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Subtasks

No response

Code of Conduct

@klion26
Copy link
Member

klion26 commented Jun 12, 2024

#2912 has enhanced the exception message by adding the file path of different file ids.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants