Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.NET out-of-proc error parsing fixes #2763

Merged
merged 3 commits into from
Mar 19, 2024
Merged

.NET out-of-proc error parsing fixes #2763

merged 3 commits into from
Mar 19, 2024

Conversation

cgillum
Copy link
Member

@cgillum cgillum commented Mar 18, 2024

Resolves #2711

There were a few edge cases that weren't being handled correctly when dealing with activity exceptions in .NET out-of-proc. The edge cases included:

  • Cases when customers override Exception.ToString() and change the exception message formatting
  • Cases when customers enable WorkerOptions.EnableUserCodeException

In the above cases, what we receive from the .NET Isolated worker changes from what we expect, resulting in exception information missing when customers try to handle exceptions in their orchestrator function code. The changes in this PR are meant to cover these corner cases and provide users with more accurate exception information.

Pull request checklist

  • My changes do not require documentation changes
    • Otherwise: Documentation PR is ready to merge and referenced in pending_docs.md
  • My changes should not be added to the release notes for the next release
    • Otherwise: I've added my notes to release_notes.md
  • My changes do not need to be backported to a previous version
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • I have added all required tests (Unit tests, E2E tests)
  • My changes do not require any extra work to be leveraged by OutOfProc SDKs
  • My changes do not change the version of the WebJobs.Extensions.DurableTask package
    • Otherwise: major or minor version updates are reflected in /src/Worker.Extensions.DurableTask/AssemblyInfo.cs
  • My changes do not add EventIds to our EventSource logs
    • Otherwise: Ensure the EventIds are within the supported range in our existing Windows infrastructure. You may validate this with a deployed app's telemetry. You may also extend the range by completing a PR such as this one.
  • My changes should be added to v3.x branch.
    • Otherwise: This change only applies to Durable Functions v2.x and will not be merged to branch v3.x.

@cgillum cgillum requested a review from davidmrdavid March 18, 2024 22:57
Copy link
Contributor

@davidmrdavid davidmrdavid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but I'd love to revisit our approach to exception serialization further (not an easy problem by any means!) since I can see there's lots of edge cases, making it tricky to reason about.

src/WebJobs.Extensions.DurableTask/OutOfProcMiddleware.cs Outdated Show resolved Hide resolved
Comment on lines +661 to +669
// "Exception of type 'ExceptionSerialization.Function+UnknownException' was thrown."
const string startMarker = "Exception of type '";
const string endMarker = "' was thrown.";
if (exception.StartsWith(startMarker) && exception.EndsWith(endMarker))
{
exceptionType = exception[startMarker.Length..^endMarker.Length];
exceptionMessage = string.Empty;
return true;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not for this PR, but I'd love to revisit this approach - something about string manipulations here makes me unsure about the robustness of our approach

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I'm uncomfortable with this too. I sent @mattchenderson a ping on Teams earlier to see if the Functions Host team might consider giving us a proper exception contract between the host and the worker in the future, which would eliminate the need for this kind of fragile string parsing.

@cgillum cgillum merged commit 20617c8 into dev Mar 19, 2024
20 checks passed
@cgillum cgillum deleted the cgillum/fix-2711 branch March 19, 2024 03:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[out-of-proc] RetryContext.LastFailure incorrectly captures thrown exception
2 participants