Add support for Open Telemetry

[tommasodotNET]

Why make this change?

Initial implementation to add open-telemetry support (reference #2397)

What is this change?

The user can add the open-telemetry configuration in the dab-config.json file as follows:

{
  "runtime": {
    ...
    "telemetry": {
      "open-telemetry": {
        "enabled": true,
        "service-name": "dab", // has a default value
        "endpoint": "OTEL_ENDPOINT",
        "headers": "x-otlp-api-key=OTEL_API_KEY" // if needed
      }
    }
    ...
  }
}

This configuration can be used alongside the current Application Insights instrumentation, to send logs, traces and metrics to both.

Currently, metrics and traces sent to OTEL comes from AspNetCore and the HttpClients.

The OpenTelemetry implementation only supports dotnet 8.

How was this tested?

• Unit Tests

[abhishekkumams]

Thanks @tommasodotNET for taking up this feature.

Added few comments.

Also,

1. Would it possible to add some integration tests that validate the complete workflow.
   and having some assertions to check that traces are created for specific operations or that metrics have the expected values?

2. For actual testing where were you checking the exported data? Can you add some information on that as well?

[tommasodotNET]

Hi @abhishekkumams. I've added some tests for the CLI. As for actual testing I have no idea at the moment. We would need an OTEL endpoint...

[abhishekkumams]

/azp run

[abhishekkumams]

@tommasodotNET , there are some formatting errors in your PR. you can run dotnet format command it will fix the failure.

[JerryNixon]

Before we can merge, we need to ensure we have a custom Activity called EntityOp.

Configuration change

This setting is obeyed regardless of host.mode value.

{
  "runtime": {
    "telemetry": {
      "open-telemetry": {
        "include-sensitive-data": <boolean, default: false>
      }
    }
  }
}

Activity lineage

#	Name	Type	Description
1	EntityOp	Activity	Entire operation.
1.1	Request	Event	Incoming HTTP/GraphQL request.
1.2	PreDatabase	Child-Activity	Pre-database operations.
1.3	Database	Event	Database query execution.
1.4	PostDatabase	Child-Activity	Post-database processing.
1.5	Response	Event	Outgoing HTTP/GraphQL response.

---

1. EntityOp (Activity)

Tags:

• entity.name
• operation.endpoint_kind rest|graphql
• operation.kind create|read|update|delete|execute
• opreation.duration_ms
• operation.success true|false

---

1.1 Request (Event)

Tags:

• http.method
• http.url ⚠️ sensative-data
• http.query_params ⚠️ sensative-data

---

1.2 PreDatabase (Activity)

The reason this is an activity is to allow for potential future Script events for pre-database operations, such as validation or transformation.

Tags:

• pre_database.script_count
• pre_database.success

1.2.1 Script (Event)

Tags:

• script.index
• script.duration_ms
• script.result

---

1.3 Database (Event)

Tags:

• db.statement ⚠️ sensative-data
• db.query.rows_returned

---

1.4 PostDatabase (Activity)

The reason this is an activity is to allow for potential future Script events for post-database processing, such as data enrichment or aggregation.

Tags:

• post_database.script_count
• post_database.success

1.4.1 Script (Event)

Tags:

• script.index
• script.duration_ms
• script.result

---

1.5 Response (Event)

Tags:

• http.status_code
• http.response.size

---

The Script (Event) entries are placeholders for future implementations. Including them now highlights why PreDatabase and PostDatabase are designed as activities—they can group and manage multiple scripts when implemented later.

[abhishekkumams]

Few Things:

1. Lets Add Custom Activity in a separate PR.
2. Can you check Telemetry Channel for OpenTelemetry, just like we have for ApplicationInsights(TelemetryTests.cs)
3. I'll take a look on your current changes as why the Logs are not being sent to OTel?

[abhishekkumams]

/azp run