Add LangFuse tracing samples - #331
Open
DABH wants to merge 6 commits into
Open
Conversation
…metry Demonstrates the recommended way to get Temporal workflow traces into Langfuse: OpenTelemetryPlugin(add_temporal_spans=True) with a replay-safe tracer provider, plus a standard OTLP/HTTP exporter pointed at Langfuse's native OpenTelemetry endpoint. No Langfuse-specific SDK or plugin needed. - ticket_triage/: LLM triage workflow (two LLM activities, a plain activity, and a human-approval update) with --replay-stress and worker-restart demos; one correctly nested Langfuse trace per run with GENERATION observations carrying model, token usage, and content. - verify_trace.py: asserts the whole observation tree, types, usage, and no-duplicates through the Langfuse public API. - langfuse/docker-compose.yml: pinned self-hosted Langfuse with headless org/project/API-key provisioning. - naive_guide_style/: deliberately broken anti-pattern (spans created in workflow code, sandbox disabled) showing duplicated, fragmented traces under replay. - RECOMMENDATION.md: customer-shareable write-up of the approach. - tests/langfuse_tracing/: CI-safe tests with mocked LLM activities, an in-memory exporter, the workflow cache disabled, and a Replayer pass asserting replay emits zero new spans.
Drop the recommendation write-up and the comparison variant, and remove references to them from the READMEs and verify_trace.py.
- Bound LLM activity retries (RetryPolicy(maximum_attempts=3)) so a misconfigured endpoint or API key fails fast instead of retrying forever, and note that each retry attempt adds its own RunActivity span; verify_trace's duplicate-row message now mentions activity retries alongside replay as a possible cause of extra spans. - Ignore .env files repo-wide so the runbook-created env file holding a real API key cannot be committed by accident. - Make the Langfuse project ID in the starter's printed trace link configurable via LANGFUSE_PROJECT_ID (defaults to the project the bundled docker-compose provisions). - Parse worker flags with argparse so typos and --help behave as expected instead of silently starting a normal-mode worker. - Replace bare os.environ lookups for Langfuse credentials with an actionable error message in telemetry.py and verify_trace.py.
The AI SDK block below already contains the identical pattern and owners, and CODEOWNERS resolves by last matching pattern, so the earlier entry was fully shadowed.
DABH
commented
Jul 28, 2026
xumaple
reviewed
Jul 29, 2026
xumaple
left a comment
xumaple
left a comment
Contributor
There was a problem hiding this comment.
Overall looks good, there were a few comments-related things that claude pulled up:
- starter.py:51 + README.md:131 — "unique workflow ID gives one Langfuse trace per run" is causally wrong. The trace ID comes from the starter's root span (starter.py:66), fresh every run regardless of workflow ID; what the unique ID buys is a unique langfuse.session.id, which is what --workflow-id resolution depends on.
- verify_trace.py:6 references a "replay-stress test" that doesn't exist — --replay-stress is a worker CLI flag. Surfaces in --help via description=doc.
- README.md:76-77 — "the identical tree" is wrong (each run is a new trace; the shape is identical), and it tells you to rerun verify_trace.py after each experiment, but neither following command block contains that invocation.
- docker-compose.yml:6-8 — "upstream binds them to 127.0.0.1" is false for MinIO (upstream publishes 9090:9000 on all interfaces), and the list omits langfuse-worker. Related: docker-compose.yml:57,65 still point Langfuse's S3 media/export external endpoints at localhost:9090, a port this file deliberately stopped publishing — any presigned media URL is a dead link (unreachable in this text-only demo).
- .env.example omits LANGFUSE_DEMO_USER, which starter.py:73 reads and verify_trace.py:208 asserts on (PLAUSIBLE — the file otherwise enumerates the full surface).
- README.md:86-87 — "kill the worker while the workflow is parked" is imprecise; early in the 20s window the workflow is still running classify_ticket, so you'd exercise activity retry instead (PLAUSIBLE).
- "Idempotent" for the LANGFUSE_INIT_* provisioning couldn't be verified offline (PLAUSIBLE).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds samples of using our OTel plugin to send traces to LangFuse from Temporal workflows.