Repository navigation
Conversation
…en't masked ctest --rerun-failed only reruns failures from the most recent ctest call. The single retry at the end only saw the parallel group's failures (and excluded the sequential tests anyway), then reset exit_code to 0 when they passed. A sequential-group failure plus any flaky parallel test went green. Retry each group immediately after it fails, and fail the step if either retry still fails. Applies to the Linux, macOS and Windows CTest steps.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
full-build.ymlruns CTest in two groups: a few resource-lockedModelFixturetests one at a time, then everything else in parallel. If either failed, it ran one retry at the end with--rerun-failedand setexit_codefrom that retry.--rerun-failedonly reruns failures from the most recent ctest call, which is the parallel group. That retry also kept the-Efilter that excludes the sequential tests.--rerun-failedmatches failures by test number, and that filter renumbers the tests, so a failed sequential test was "rerun" as whichever test had its number in the filtered list (the second mechanism in #5660). So if a sequential test failed and any parallel test flaked in the same run, the retry ran only the flaky test, it passed, and the step reportedexit_code=0.This retries each group right after it fails and fails the step if either retry still fails. The change is the same in the Linux, macOS and Windows CTest steps.
Fixes #5660. Found while reviewing #5681. After that PR, Full Build is the only macOS/Windows test gate on PRs, so a masked failure there merges.
Testing
I ran the macOS step script, extracted from the workflow, against a three-test CTest project (one sequential-group test, one flaky parallel test, one passing test):
exit_code=0exit_code=808000With the sequential test moved to the end of the test list (so the renumbering applies), the old script also hid a sequential failure with no other failures (
always fails/passes: before0, after8). The new script gave the expected result in every case in both orders.I didn't run the PowerShell (Windows) version locally; its logic matches the bash version. actionlint reports no new issues (37 → 35).
🤖 Generated with Claude Code