Skip to content

Delete empty TsFile sidecars after LOAD - #18599

Open
luoluoyuyu wants to merge 1 commit into
apache:masterfrom
luoluoyuyu:codex/delete-empty-tsfile-after-load
Open

Delete empty TsFile sidecars after LOAD#18599
luoluoyuyu wants to merge 1 commit into
apache:masterfrom
luoluoyuyu:codex/delete-empty-tsfile-after-load

Conversation

@luoluoyuyu

Copy link
Copy Markdown
Member

Description

When LOAD skips an empty TsFile with on-success=delete, also remove its resource and mods files, and add an integration test covering the deletion.


This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR

When LOAD skips an empty TsFile with on-success=delete, also remove its resource and mods files, and add an integration test covering the deletion.

@Caideyipi Caideyipi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已审阅,未发现阻塞问题。

实现与现有 LoadSingleTsFileNode.clean() 的清理语义一致:分别重试删除 TsFile、.resource.mods2.mods,并且单个文件删除失败不会阻止后续 sidecar 的清理。新增集成测试覆盖了空 TsFile 使用 on-success=delete 时四类文件均被删除的行为。

本地验证:

  • spotless:check(DataNode 和 integration-test)通过;
  • IoTDBLoadTsFileIT#testDeleteEmptyTsFileAfterLoad 通过(1 test,0 failures,0 errors);
  • 完整依赖 reactor 构建成功。

当前唯一失败的 CI 是未涉及本 PR 变更路径的 IoTDBPipeAutoDropIT.testAutoDropIgnoredUnmatchedDataRegions 数据一致性波动;其余已完成检查均通过,包括 Windows、unit tests 和 zh-locale-compile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants