Skip to content

[fix](case) lower cold compaction threshold in fault injection test#65554

Open
shuke987 wants to merge 1 commit into
apache:masterfrom
shuke987:codex/fix-cold-data-compaction-threshold
Open

[fix](case) lower cold compaction threshold in fault injection test#65554
shuke987 wants to merge 1 commit into
apache:masterfrom
shuke987:codex/fix-cold-data-compaction-threshold

Conversation

@shuke987

Copy link
Copy Markdown
Collaborator

What problem does this PR solve?

Issue Number: N/A

Related PR: #55333

Problem Summary:

test_cold_data_compaction_fault_injection creates five remote rowsets, but the current default cold_data_compaction_score_threshold is 100. The tablet's cold-data compaction score is 5, so the producer keeps skipping it and the case times out while the remote object count stays at 6.

Temporarily set cold_data_compaction_score_threshold to 4 for this case, matching the existing cold_heat_separation/cold_data_compaction.groovy coverage. setBeConfigTemporary restores the original value after the case.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

    Manual validation:

    • Branch-4.1 test cluster single-case run: PASS 1/1.
    • Remote object count changed from 6 to 7 after one 60-second producer interval.
    • All four BE thresholds were restored from the temporary value 4 to 100.
    • Debug-point cleanup completed.
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@shuke987

Copy link
Copy Markdown
Collaborator Author

/review

@shuke987

Copy link
Copy Markdown
Collaborator Author

run buildall

@github-actions

Copy link
Copy Markdown
Contributor

Codex automated review failed and did not complete.

Error: You've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Jul 20th, 2026 12:23 AM.
Workflow run: https://github.com/apache/doris/actions/runs/29251649365

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

@shuke987

Copy link
Copy Markdown
Collaborator Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review completed for PR 65554.

I found two blocking issues in the test update. First, the new setBeConfigTemporary path may fail before applying or restoring cold_data_compaction_score_threshold because the BE update-config handler dereferences a missing optional persist parameter. Second, once the threshold is applied, the test can still pass on a normal successful cold-data compaction path because the enabled debug point is not on the cold-data score/execute path and the object count of 7 is not unique fault evidence.

Critical checkpoint conclusions: goal/test proof is not satisfied yet because the modified regression test can fail before setup or false-pass without proving the intended fault; the change is narrow but relies on shared BE config and cold-compaction behavior; no new production concurrency was added, but the mutable process-wide config is only safe if the update/restore API works; no storage-format or rolling-upgrade compatibility change is introduced; the relevant parallel successful cold-data-compaction path was checked and overlaps the asserted object count; test coverage is present as a regression test edit, but I did not run tests or builds because this review environment explicitly forbids builds/code execution beyond review operations. User focus: no additional user-provided focus was supplied.

def tabletName = "test_cold_data_compaction_fault_injection"

def customBeConfig = [
cold_data_compaction_score_threshold : 4

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lowering the threshold does make the producer submit this tablet, but it also means the test can pass without exercising the intended fault. The producer uses calc_cold_data_compaction_score() and skips only score < cold_data_compaction_score_threshold; with five cooled DUP-key rowsets the score is 5, so threshold 4 admits the tablet. The enabled debug point is in _calc_cumulative_compaction_score(), which is not used by the cold-data producer or by ColdDataCompaction::prepare_compact()/execute_compact(). A normal successful cold compaction can still leave the five old remote rowset objects plus one new output rowset plus the stable meta object until remote cleanup runs, so the size() == 7 wait can succeed on the non-fault path. Please inject a point that is actually reached by cold-data compaction after the output is written, or assert a state that cannot occur after a successful cold compaction.

cold_data_compaction_score_threshold : 4
]

setBeConfigTemporary(customBeConfig) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new wrapper relies on /api/update_config before the test can lower the threshold, but the helper posts only ?<param>=<value> and omits persist. In the current BE handler, persist is described as optional, yet handle_update_config() unconditionally does req->params()->find("persist")->second.compare("true") as soon as any query parameter exists. Since set_be_param() sends /api/update_config?cold_data_compaction_score_threshold=4, the handler can dereference end() before applying the config, and the finally restore path uses the same request shape. Please either make the update handler use req->param("persist")/an iterator check, or include persist=false in the regression helper before relying on this wrapper here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants