Limited OTLP exporter request authentication methods

[majanjua-amzn]

What are you trying to achieve?

We are trying to allow the OTLP exporter to be able to authenticate dynamically to OTLP endpoints that require more than static credentials like API keys in the environment.

What did you expect to see?

We expected to see a mechanism for users of OTel to be able to interact with the HTTP client used in the OTLP exporter and/or modify the HTTP headers as the requests are sent to an OTLP endpoint, however, neither of these mechanisms exists. This capability is missing in all supported OTel instrumentation languages we've looked into (Java, Python, JS, .NET).

Additional context

This discussion was originally triggered by requests to have AWS SigV4 authentication in the OTel Java instrumentation (open-telemetry/opentelemetry-java/issues/7002). In that discussion, we found the capabilities missing in Java are also missing in the other languages, that the issue is not limited to AWS SigV4 authentication, and that dynamic authorization mechanisms for data sent to OTLP endpoints were not supported by OTel instrumentations.

As such, we brought this issue up in the OpenTelemetry Sig: Specifications meeting on 28/01/2025 (Google Doc, meeting recording). In this meeting, we discussed the issue, finding that there is some community interest to address this issue, discussing some potential solutions, and gathering some potential first steps (namely, creating this ticket).

In that meeting, the following related issue previously raised in the golang instrumentation was brought up: open-telemetry/opentelemetry-go/issues/2632. This related issue describes a similar lacking capability in the instrumentation in that golang users are unable to specify the HTTP client used to export data to the OTLP endpoint. We can see in that issue there is interest from the community and other stakeholders like Azure (comment being referenced) to be able to accommodate more complex HTTP request manipulation, especially for the purposes of secure authentication.

One discussed potential solution that is of note for future reference is to introduce callback hooks in the exporters to allow for response-driven actions like re-authentication to the OTLP endpoint, e.g. make request → 401 response → callback hook triggers reauthentication.

[dmathieu]

Discussion on the same subject in the Go repository.
open-telemetry/opentelemetry-go#4536

[brunobat]

@trask, we don't have official support for AWS SigV4 authentication in Quarkus.
Users would have to connect to aws following the examples in the docs: https://github.com/aws-samples/sigv4a-signing-examples/tree/main/java

[srprash]

Hi @brunobat This is helpful.

However the issue here is that the clients making the call to an OTLP endpoint is abstracted away within the exporters. So it's not possible for OTel users to modify/enhance the behavior of these clients to add a complex auth process like AWS SigV4 where the entire request, including the body, is needed to construct the signature.