Is there any plan for OpenTelemetry instrumentation?

[febus982]

Having tracing, metrics and logs supported for OpenTelemetry would be a great addition.

[s3rius]

Hello. I think it would be great to have a taskiq instrumentation, but as I understand taskiq already has a simple integration with otlp out of the box. Since we use mainstream libraries for brokers, you can enable instrumentation for those libraries to get all the traces.

[nickderobertis]

We are currently configuring tracing throughout our application, instrumenting the libraries powering brokers helps but the main thing missing here from our perspective is root spans that lasts the lifecycle of a task. Theoretically it should be possible to have the producer of the task start a trace and it be connected to the worker handling the task in a single trace. It seems like the instrumentation for Celery accomplishes this.

Beyond cross-service tracing, not having a root span at the framework level seems to be causing issues for us. We are using FastAPI as our API framework and the instrumentation for that sets a root span for every request automatically. Without having this at the framework level, traces end up without root spans and so we have some traces lasting close to an hour with TaskIQ. Maybe there is a way to fix this on the otel/Tempo side (we are not experts with otel) but everything works as expected for our FastAPI application.

As a workaround for now we've added spans to task workers by wrapping broker.task in our own custom decorator. This seems to allow us to trace our task execution in the worker accurately but we still end up with traces without root spans containing many unrelated spans over a long duration.

[s3rius]

Okay, I got the point. We will look into it.

[ipfans]

It would be better if logfire could be easily integrated. Thank you guys

[febus982]

Logfire is an OpenTelemetry implementation. Instrumenting OpenTelemetry would automatically work on logfire.

[karansharmapaypay]

Hi, any updates here? I was considering using taskiq for our usecase (currently celery+fastapi). This seems to be a deal breaker for us since we rely heavily on end to end traces for a lot of our monitoring.

[febus982]

I ended up using another package :/

[s3rius]

There is a plan, but right now I don't have much resources to spare on that. I would be really glad if you could help with that. Right now we're in the middle of preparing some big updates.

[ipfans]

@s3rius Could you share how to implement it? I would like to share the idea if possible, and the code can be implemented by the community.

[ktnlvr]

Hey! What’s the current roadblock with integrating OpenTelemetry reporting? From my brief look, it seems that simply supplying the trace ID to an executing task would be enough.

Regarding the issue brought up by @nickderobertis: from what I know, OpenTelemetry does not allow suspending spans. So having one span last several hours is actually expected. You can always scope down to a more specific subspan when analyzing. Also, if spans are implemented, the loggers should automatically pick up on that. That way, we get two features for the price of one.

I chose TaskIQ over Celery because it allows for easy async. However, not having telemetry in a distributed system could be a dealbreaker. I’ll need it for a personal project anyway, so I can start working on the basic instrumentation design if that would be helpful.