Skip to content

[SPARK-58734][INFRA] Skip a declined cherry-pick instead of aborting the merge in merge_spark_pr.py - #57959

Closed
uros-b wants to merge 2 commits into
apache:masterfrom
uros-b:merge-script-cherrypick-skip
Closed

[SPARK-58734][INFRA] Skip a declined cherry-pick instead of aborting the merge in merge_spark_pr.py#57959
uros-b wants to merge 2 commits into
apache:masterfrom
uros-b:merge-script-cherrypick-skip

Conversation

@uros-b

@uros-b uros-b commented Aug 12, 2026

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

This PR changes how dev/merge_spark_pr.py handles a committer declining to resolve a conflicting backport cherry-pick.

Previously, when a cherry-pick onto a maintenance branch conflicted, the script prompted Would you like to manually fix-up this merge?. Answering N went through continue_maybe(..., cherry=True), which aborted the cherry-pick and then called fail("Okay, exiting") — terminating the whole run via sys.exit(-1). As a result the committer never reached the JIRA-resolution step, and the process exited non-zero, even though the merge into the target branch (and any earlier cherry-picks) had already been pushed.

This PR makes declining a cherry-pick fix-up skip only that one branch and continue:

  • Add a SkipCherryPick exception. When a cherry-pick fix-up prompt is declined, continue_maybe aborts the cherry-pick, restores the working tree (clean_up()), and raises SkipCherryPick instead of calling fail().
  • cherry_pick() catches SkipCherryPick and returns only the picks that actually landed (empty, or — in the Upstream-First two-branch path — just the sibling branch that was already pushed).
  • The two merge/backport loops in main() already consume the returned list generically (merged_refs + [] is a no-op), so no loop changes are needed: after a skip they simply offer the next branch and still proceed to resolve the associated JIRA.

Hard aborts elsewhere are unchanged (e.g. declining the push prompt, or choosing [a]bort at the Upstream-First prompt, still exit).

Why are the changes needed?

Backport cherry-pick conflicts are routine, and by the time one occurs the merge into the target branch has already been pushed. Aborting the entire script on a declined fix-up means the committer:

  • skips JIRA resolution (the ticket is left Open, which is easy to miss and tedious to reconcile after the fact), and
  • gets a non-zero exit for what is a normal "do not backport that one branch" decision.

Declining a single conflicting backport should skip just that branch and let the merge finish cleanly.

Does this PR introduce any user-facing change?

No. This changes a committer-only developer tool (dev/merge_spark_pr.py); it is not part of any Spark release artifact.

How was this patch tested?

  • The module's inline doctests still pass (run by doctest.testmod() at startup): 76 attempted, 0 failed.
  • python3 -m py_compile dev/merge_spark_pr.py is clean.
  • Manually traced the affected control-flow paths:
    • single-branch pick declined -> cherry_pick returns [], the loop re-prompts and JIRA resolution still runs;
    • Upstream-First [b]oth path with the second pick declined -> the already-pushed sibling branch is still returned and recorded in the merge comment / merged_refs;
    • declining non-cherry prompts, and [a]bort at the Upstream-First prompt, still hard-exit as before.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code with Claude Opus 4.8

…the merge in merge_spark_pr.py

When a backport cherry-pick conflicts, declining the "Would you like to
manually fix-up this merge?" prompt aborted the entire script via fail(),
so the committer never reached the JIRA-resolution step (and the run
exited non-zero) -- even though the merge into the target branch had
already been pushed. Backport conflicts are routine, so declining one
should skip only that branch.

Add a SkipCherryPick exception, raised (after aborting the cherry-pick
and restoring the tree) when a cherry-pick fix-up is declined, and catch
it in cherry_pick() so it returns only what actually landed. The merge
loops already treat the returned list generically, so they keep offering
further branches and still resolve the associated JIRA.

Generated-by: Claude Code with Claude Opus 4.8
@uros-b
uros-b requested a review from cloud-fan August 12, 2026 11:06

@uros-b uros-b left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cloud-fan Please review.

@cloud-fan cloud-fan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 blocking, 0 non-blocking, 2 nits.
The control flow is sound; two added comments need small accuracy and grammar fixes.

Nits: 2 minor items (see inline comments).

Verification

Traced both main cherry-pick loops through the single-branch and Upstream-First paths. Returned branch lists preserve successful pushes, skipped branches are not added to merged_refs, and the normal merge path still reaches JIRA resolution.

Comment thread dev/merge_spark_pr.py Outdated
Comment thread dev/merge_spark_pr.py Outdated
@uros-b

uros-b commented Aug 13, 2026

Copy link
Copy Markdown
Member Author

Thank you @cloud-fan and @HyukjinKwon!

@uros-b uros-b closed this in d23432e Aug 13, 2026
@uros-b

uros-b commented Aug 13, 2026

Copy link
Copy Markdown
Member Author

Merge Summary:

Posted by merge_spark_pr.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants