Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add gRPC span and correlate client and orchestrator functions for DF .NET Isolated Distributed Tracing #2998

Open
wants to merge 3 commits into
base: dev
Choose a base branch
from

Conversation

bachuv
Copy link
Collaborator

@bachuv bachuv commented Dec 17, 2024

These changes help correlate client and orchestrator functions in DF .NET Isolated. It includes a new "gRPC start orchestration" span.

Gantt chart (shows HTTP trigger and orchestrator function in one trace)
image

Pull request checklist

  • My changes do not require documentation changes
    • Otherwise: Documentation PR is ready to merge and referenced in pending_docs.md
  • My changes should not be added to the release notes for the next release
    • Otherwise: I've added my notes to release_notes.md
  • My changes do not need to be backported to a previous version
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • I have added all required tests (Unit tests, E2E tests)
  • My changes do not require any extra work to be leveraged by OutOfProc SDKs
    • Otherwise: That work is being tracked here: #issue_or_pr_in_each_sdk
  • My changes do not change the version of the WebJobs.Extensions.DurableTask package
    • Otherwise: major or minor version updates are reflected in /src/Worker.Extensions.DurableTask/AssemblyInfo.cs
  • My changes do not add EventIds to our EventSource logs
    • Otherwise: Ensure the EventIds are within the supported range in our existing Windows infrastructure. You may validate this with a deployed app's telemetry. You may also extend the range by completing a PR such as this one.
  • My changes should be added to v3.x branch.
    • Otherwise: This change only applies to Durable Functions v2.x and will not be merged to branch v3.x.

@@ -154,8 +155,22 @@ public override Task<Empty> Hello(Empty request, ServerCallContext context)
{
try
{
ActivitySource activityTraceSource = new ActivitySource("DurableTask.WebJobs");
Copy link
Collaborator Author

@bachuv bachuv Dec 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For context, the source name is "DurableTask.WebJobs" because we load the DurableTelemetryModule and it looks for sources that start with "DurableTask" (related code link). I'm working on updating the listeners to add a new one ("WebJobs.Extensions.DurableTask") when we initialize the module so we can remove this "DurableTask.WebJobs" that's just used to test for now.

Comment on lines +80 to +84
Activity? currActivity = Activity.Current;
if (options == null && options?.ParentTraceId == null && currActivity != null)
{
options = new StartOrchestrationOptions(ParentTraceId: currActivity?.Id?.ToString());
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the ParentTraceId information to the StartOrchestrationOptions, but I'm wondering if there's a better place to store this information since StartOrchestrationOptions can be configured by the customer and we don't necessarily need customers to have access to ParentTraceId.

Copy link
Contributor

@jviau jviau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to avoid having a new activity source and span added. We should try and having the trace ID captured from the worker directly added to the enqueued orchestration. I think to accomplish this we will want to hydrate a shim activity which has its ID set to the trace context from the gRPC payload.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants