-
Notifications
You must be signed in to change notification settings - Fork 14.9k
MINOR: Remove onPartitionsDeleted from GroupCoordinator interface #21263
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This patch removes the `onPartitionsDeleted` method from the `GroupCoordinator` interface by moving its functionality into `onMetadataUpdate`. The `MetadataDelta` already contains the deleted topic information via `delta.topicsDelta().deletedTopicIds()`, making the separate method redundant.
| // Wait on the results. | ||
| CompletableFuture.allOf(futures.toArray(new CompletableFuture<?>[0])); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is a but here as the code does not actually waits on the future to complete.
squah-confluent
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the patch!
Not introduced in this PR but I'm not too happy about the onMetadataUpdate being the only method that throws ExecutionException and InteruptedException in the interface. Based on https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Future.html#get--, ExecutionException should never happen since we catch exceptions coming from the futures and maybe we could swallow InterruptedException and convert it to Thread.currentThread().interrupt()?
| } | ||
|
|
||
| @Test | ||
| public void testOnPartitionsDeletedWhenServiceIsNotStarted() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we have an equivalent test for onMetadataUpdate? I couldn't find one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added it. good catch.
Interesting... An alternative would be to use |
That would work nicely, we could use that instead. |
| case t: Throwable => metadataPublishingFaultHandler.handleFault("Error updating group " + | ||
| s"coordinator with deleted partitions in $deltaName", t) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems to be replaced by the general with local changes but I guess it's fine as we also log error message in futures exception
dongnuo123
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR! Lgtm.
|
|
||
| @Test | ||
| public void testCompleteTransactionWhenNotCoordinatorServiceStarted() { | ||
| public void testCompleteTransactionWhenNotCoordinatorServiceStarted() throws ExecutionException, InterruptedException { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for updating the PR. Did we miss removing these?
|
|
||
| @Test | ||
| public void testPersisterInitializeSuccess() { | ||
| public void testPersisterInitializeSuccess() throws ExecutionException, InterruptedException { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We missed removing these down here too
lianetm
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Just a comment on test
| // Verify method is blocked. | ||
| assertFalse(resultFuture.isDone()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this enough to be sure that we're blocking? I guess the future here could be incomplete only because the async execution of onMetadataUpdate hasn't made it til the end where it's supposed to block/join on the write operations (not because it is blocked waiting on the operations completion). If so, this wouldn't catch a bug like the one you pointed out above with the missing join().
What about we verify that we did schedule the operations before this assert?
~ verify(runtime, timeout(xxx).times(2)).scheduleWriteAllOperation) + assertFalse(resultFuture.isDone())
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good callout. Addressed it.
lianetm
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! LGTM
This patch removes the
onPartitionsDeletedmethod from theGroupCoordinatorinterface by moving its functionality intoonMetadataUpdate. TheMetadataDeltaalready contains the deletedtopic information via
delta.topicsDelta().deletedTopicIds(), makingthe separate method redundant.
Reviewers: Dongnuo Lyu [email protected], Sean Quah
[email protected], Lianet Magrans [email protected]