Skip to content

Prevent local data deletion until tiering completes #1919

@luoyuxia

Description

@luoyuxia

Search before asking

  • I searched in the issues and found nothing similar.

Description

Currently, if a table fails to dump data to Paimon within 2 minutes due to various reasons (network issues, Paimon service exceptions, etc.), the data on local disk will still be deleted when TTL expires, which may cause data loss.

Current Behavior

  • Local disk data is deleted strictly according to TTL expiration time
  • Data will be deleted even if tiering (dumping to Paimon) has not completed
  • This may lead to data loss risk

Expected Behavior

  • Local disk data should only be deleted according to TTL after successful tiering (dumping to Paimon)
  • If data has not completed tiering, it should be retained even after TTL expiration until tiering is complete

Willingness to contribute

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions