Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GOBBLIN-2134] update job status to SKIPPED for all the dependent jobs of a cancelled job #4029

Closed
wants to merge 2 commits into from

Conversation

arjun4084346
Copy link
Contributor

@arjun4084346 arjun4084346 commented Aug 15, 2024

Dear Gobblin maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!

JIRA

Description

  • Here are some details about my PR, including screenshots (if applicable):
    created SKIPPED execution status
    used it for the jobs that cannot be run because it's parent job is cancelled

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason:
    updated tests in KillDagProcTest

Commits

  • My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

@arjun4084346 arjun4084346 changed the title Dpemonitoringfix3.2 [GOBBLIN-2134] update job status to SKIPPED for all the dependent jobs of a cancelled job Aug 15, 2024
@@ -215,8 +216,6 @@ public void killDagNode() throws IOException, URISyntaxException, InterruptedExc
.getInvocations()
.stream()
.filter(a -> a.getMethod().getName().equals("cancelJob"))
.filter(a -> ((Properties) a.getArgument(1))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the first job was launched, but we are cancelling the 3rd job, so the cancelParams will not have "SPEC_PRODUCER_SERIALIZED_FUTURE". this is realized when i fixed the test mock behavior above

Copy link
Contributor

@Will-Lo Will-Lo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to test this thoroughly with unit tests, I believe this can cause the JobStatusMonitor to have some issues since it doesn't have SKIPPED in the JobStatusMonitor ordered execution statuses?

@arjun4084346
Copy link
Contributor Author

Need to test this thoroughly with unit tests, I believe this can cause the JobStatusMonitor to have some issues since it doesn't have SKIPPED in the JobStatusMonitor ordered execution statuses?

This change is not emitting SKIPPED event, so JobStatusMonitor will never get any such event. It is just updating the status in DagNode. But yes, let me update the code to actually emit the event so get job status call show the job status as SKIPPED; and incorporate it in ORDERED_EXECUTION_STATUS

@arjun4084346 arjun4084346 force-pushed the dpemonitoringfix3.2 branch 2 times, most recently from c05a3c4 to 3938e0c Compare September 4, 2024 19:07
@arjun4084346 arjun4084346 force-pushed the dpemonitoringfix3.2 branch 2 times, most recently from 81cfc74 to 4722928 Compare September 5, 2024 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants