Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Execute nexus operation from a workflow #1473

Merged

Conversation

bergundy
Copy link
Member

@bergundy bergundy commented May 15, 2024

What was changed

Added the ability to execute nexus operations from a workflow.
This PR is stacked on top of #1466.

Checklist

@bergundy bergundy requested a review from a team as a code owner May 15, 2024 21:45
Copy link
Member

@cretz cretz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, not much to add here. I didn't look into the guts of the state machine stuff, seems to be copied from child workflow. I hope we have enough tests to cover most angles there especially with regards to cancel before/during/after start/finish.

internal/workflow.go Show resolved Hide resolved
workflow/workflow.go Outdated Show resolved Hide resolved
internal/error.go Show resolved Hide resolved
internal/error.go Show resolved Hide resolved
internal/failure_converter.go Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May not be needed now, but may want to make an issue to ensure tracing interceptor is updated to support this (and check context propagation).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1474

context propagation may come later too, for now I'm more concerned with nexus headers (map[string]string).

eventAttributes := e.GetNexusOperationScheduledEventAttributes()
commandAttributes := d.GetScheduleNexusOperationCommandAttributes()

if eventAttributes.GetService() != commandAttributes.GetService() || eventAttributes.GetOperation() != commandAttributes.GetOperation() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To confirm, we allow endpoint to be changed without failing determinism but not service or operation? One might expect service and endpoint to be the same kind of thing, or at least it may be confusing to explain to users that half of the Nexus client creation can be changed and the other half cannot. I think it should be both endpoint and service or neither. An argument for both is that endpoints can be changed server side.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Endpoints are env specific, so I'd rather allow this to be changed as long as the "type" is preserved. It's the same as us not checking task queue for activity.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unresolving to continue conversation. I think you should either allow the two things that create a client to be changed or disallow the two things that create a client from being changed. Having one of those two changeable is confusing. If you're looking for a task queue analogy, I think that's endpoint + service and if we had an "activity client", creating it would probably have the task queue like the nexus client has endpoint + service.

internal/workflow.go Show resolved Hide resolved
internal/workflow.go Show resolved Hide resolved
internal/workflow.go Outdated Show resolved Hide resolved
Copy link
Member Author

@bergundy bergundy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressing comments shortly.

eventAttributes := e.GetNexusOperationScheduledEventAttributes()
commandAttributes := d.GetScheduleNexusOperationCommandAttributes()

if eventAttributes.GetService() != commandAttributes.GetService() || eventAttributes.GetOperation() != commandAttributes.GetOperation() {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Endpoints are env specific, so I'd rather allow this to be changed as long as the "type" is preserved. It's the same as us not checking task queue for activity.

internal/internal_task_handlers.go Outdated Show resolved Hide resolved
internal/error.go Show resolved Hide resolved
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1474

context propagation may come later too, for now I'm more concerned with nexus headers (map[string]string).

internal/workflow.go Show resolved Hide resolved
internal/workflow.go Outdated Show resolved Hide resolved
workflow/workflow.go Outdated Show resolved Hide resolved
internal/internal_workflow_testsuite.go Outdated Show resolved Hide resolved
@bergundy bergundy force-pushed the bergundy/nexus-operation-from-workflow branch from 7b5d3b9 to dcfa8cf Compare May 17, 2024 00:31
@bergundy
Copy link
Member Author

@cretz @Quinn-With-Two-Ns anything preventing your approval?

Copy link
Member

@cretz cretz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was hoping to continue the conversation about having only half of the params for making a new nexus client be subject to determinism checks. But maybe another time.

Copy link
Contributor

@Quinn-With-Two-Ns Quinn-With-Two-Ns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was waiting for test suite support in this PR, but doing it in a separate PR works to.

* Add test environment support for Nexus Operations

* Change client to not allow any direct calls
@bergundy bergundy merged commit 646c0a1 into bergundy/nexus-handler Jun 19, 2024
4 of 12 checks passed
@bergundy bergundy deleted the bergundy/nexus-operation-from-workflow branch June 19, 2024 23:37
@bergundy bergundy mentioned this pull request Jun 19, 2024
2 tasks
@bergundy bergundy mentioned this pull request Jul 19, 2024
bergundy added a commit that referenced this pull request Jul 22, 2024
## What was changed

- Added the `temporalnexus` package and implemented the handler side for Nexus, including registering and dispatching Nexus Operations.
- Added the ability to execute Nexus Operations from a workflow.
- Added basic support for running Nexus Operations in the test environment.
- Added memoizing to `worker.Start()` to return consistent errors to callers and avoid rerunning the function unnecessarily.
- Updated the integration test's dev server to run CLI `0.14.0-nexus.0` which includes server `1.25.0-rc.0`.

See the [proposal](https://github.com/temporalio/proposals/blob/b72c49b0c2278e916265b00a49638006f8fce469/nexus/sdk-go.md) for more information.

Most of this code has been reviewed already in #1466, #1473, and #1475, which are all squashed in the first commit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants