Skip to content

OpenTelemetry spans missing workflow input/output attributes #1190

@maver1ck

Description

@maver1ck

Summary

temporalio.contrib.opentelemetry.TracingInterceptor does not put workflow input/output values on the spans it exports. Only the workflow/run IDs show up, so downstream OTEL backends (Langfuse in my case) cannot see what was invoked.

Reproduction

  1. Save the script below as temporal_minimal_otel_bug.py (needs a Temporal dev server on localhost:7233.
"""Minimal Temporal workflow runner with optional OpenTelemetry tracing."""

from __future__ import annotations

import asyncio
import logging
import os
import uuid

from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor
from temporalio import workflow
from temporalio.client import Client
from temporalio.worker import Worker
from temporalio.contrib.opentelemetry import TracingInterceptor


log = logging.getLogger(__name__)


def _otel_interceptors() -> list[object]:
    """Return Temporal interceptors that emit OpenTelemetry spans when available."""
    if TracingInterceptor is None:
        log.debug(
            "Skipping OpenTelemetry tracing; install temporalio[opentelemetry] extras to enable."
        )
        return []
    log.info("OpenTelemetry tracing enabled for minimal PoC (console exporter).")
    return [TracingInterceptor()]


def _configure_console_exporter() -> None:
    """Set up a console exporter so spans are visible locally."""
    existing = trace.get_tracer_provider()
    if isinstance(existing, TracerProvider):
        # Already configured (e.g., tests) -> skip
        return
    resource = Resource.create({"service.name": os.getenv("OTEL_SERVICE_NAME", "temporal-minimal")})
    provider = TracerProvider(resource=resource)
    provider.add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter()))
    trace.set_tracer_provider(provider)


@workflow.defn(sandboxed=False)
class HelloWorkflow:
    @workflow.run
    async def run(self, name: str) -> str:
        log.info("Running HelloWorkflow for %s", name)
        return f"Hello, {name}!"


async def main() -> None:
    _configure_console_exporter()
    interceptors = _otel_interceptors()
    client = await Client.connect(
        "localhost:7233",
        interceptors=interceptors,
    )

    async with Worker(
        client,
        task_queue="minimal",
        workflows=[HelloWorkflow],
    ):
        result = await client.execute_workflow(
            HelloWorkflow.run,
            "Temporal",
            id=f"hello-{uuid.uuid4()}",
            task_queue="minimal",
        )
        print(result)


if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)
    asyncio.run(main())
  1. Run uv run python temporal_minimal_otel_bug.py.
  2. Check stdout for Temporal spans.

Expected

Workflow spans include attributes for the serialized input and output (even truncated strings would work) so downstream tools can show what the workflow did.

Actual

Only identifiers are exported; input/output payloads are absent even though the Temporal history contains them.

Image

Environment

  • temporalio==1.18.0
  • opentelemetry-sdk==1.38.0
  • Python 3.12
  • Temporal Server 2.41.0
  • OTLP exporter: HTTP/protobuf to Langfuse Cloud

Notes

Temporal history for the same run shows one input payload and one output payload The data is available; it just never makes it onto the spans.

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions