Skip to content

[ISSUE #10383] Fix flaky store and proxy tests#10407

Open
RongtongJin wants to merge 2 commits into
developfrom
codex/fix-store-flaky-tests
Open

[ISSUE #10383] Fix flaky store and proxy tests#10407
RongtongJin wants to merge 2 commits into
developfrom
codex/fix-store-flaky-tests

Conversation

@RongtongJin
Copy link
Copy Markdown
Contributor

@RongtongJin RongtongJin commented May 29, 2026

Summary

This is a follow-up for the tests called out around #10404 and the broader #10383 test stability/dependency work. It keeps the tests enabled and makes the repeated-run behavior deterministic instead of quarantining them.

  • Use dynamic HA ports in AutoSwitchHATest and pass the actual bound HA address after startup.
  • Wait for HA replication connection establishment before producing messages after master/slave role changes.
  • Avoid double shutdown/destroy in AutoSwitchHATest teardown paths to prevent repeated-run file lock noise.
  • Use dynamically allocated DLedger ports (localhost:0 for single-node tests, available ephemeral ports for two-node tests) instead of the shared nextPort() sequence.
  • Make ReceiptHandleProcessorTest deterministic by setting the mocked channel online before the renewal scheduler starts, returning a completed renewal future, and shutting down the processor in teardown.

Validation

  • Reproduced AutoSwitchHATest#testAddBroker failure with 127.0.0.1:10912 pre-bound; it failed before the change and passed after the dynamic-port fix.
  • mvn -q -pl store -am -DskipITs -DfailIfNoTests=false -Dcheckstyle.skip=true -Dspotbugs.skip=true -Dtest=AutoSwitchHATest#testConfirmOffset test passed 3 consecutive runs.
  • mvn -q -pl store -am -DskipITs -DfailIfNoTests=false -Dcheckstyle.skip=true -Dspotbugs.skip=true -Dtest=AutoSwitchHATest,DLedgerCommitlogTest,DLedgerMultiPathTest test passed 100 total runs; each run reported Tests run: 12, Failures: 0, Errors: 0, Skipped: 3 on this macOS environment, with no ConditionTimeoutException, BindException, release file lock error, ClassNotFoundException, or forked VM failure in the accepted logs.
  • mvn -q -pl store -am -DskipITs -DfailIfNoTests=false -Dcheckstyle.skip=true -Dspotbugs.skip=true -Dtest=DLedgerCommitlogTest,DLedgerMultiPathTest test passed on this macOS environment with the existing platform assumptions (Tests run: 2, Failures: 0, Errors: 0, Skipped: 2).
  • mvn -q -pl proxy -am -DskipITs -DfailIfNoTests=false -Dcheckstyle.skip=true -Dspotbugs.skip=true -Dtest=ReceiptHandleProcessorTest test passed 100 consecutive independent Maven runs, with no Argument(s) are different, NullPointerException, or Mockito strict stubbing error in the logs.
  • mvn -q -pl store -am -DskipITs -Dcheckstyle.skip=true -Dspotbugs.skip=true test-compile passed.

Bazel validation was not run locally because bazel is not installed in this environment (zsh: command not found: bazel).

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 29, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 47.92%. Comparing base (41a3a35) to head (e02728f).
⚠️ Report is 3 commits behind head on develop.

Additional details and impacted files
@@              Coverage Diff              @@
##             develop   #10407      +/-   ##
=============================================
- Coverage      49.01%   47.92%   -1.09%     
+ Complexity     13485    13249     -236     
=============================================
  Files           1376     1376              
  Lines         100527   100536       +9     
  Branches       12983    12983              
=============================================
- Hits           49274    48184    -1090     
- Misses         45251    46410    +1159     
+ Partials        6002     5942      -60     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@RongtongJin RongtongJin changed the title [codex] Fix flaky store HA and DLedger tests [ISSUE #10383] Fix flaky store HA and DLedger tests May 29, 2026
@RongtongJin RongtongJin changed the title [ISSUE #10383] Fix flaky store HA and DLedger tests [ISSUE #10383] Fix flaky store and proxy tests May 29, 2026
@RongtongJin RongtongJin marked this pull request as ready for review May 30, 2026 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants