feat(integrations): langfuse integration #1
Conversation
![cubic-dev-local[bot]](https://avatars.githubusercontent.com/u/189440881?s=60&v=4){kind=small}
- Added Langfuse configuration to track observability metrics. - Introduced for optional keys. - Enhanced agent functions to include Langfuse instrumentation and trace updates. - Updated dependencies to include .
Signed-off-by: Chris Lo <[email protected]>
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
4 issues found across 6 files
Prompt for AI agents (all 4 issues)
Understand the root cause of the following 4 issues and fix them.
<file name="packages/tracecat-registry/tracecat_registry/integrations/pydantic_ai.py">
<violation number="1" location="packages/tracecat-registry/tracecat_registry/integrations/pydantic_ai.py:334">
Instrumentation is always enabled. This can cause unnecessary initialization or errors if Langfuse is not configured. Make it conditional on configured secrets.
DEV MODE: This violation would have been filtered out by GPT-5.
Reasoning:
• **GPT-5**: instrument=True on pydantic_ai.Agent does not auto-enable Langfuse; without telemetry configuration it’s a no-op. Secrets are optional, so the claimed risk is speculative and low impact.
• **Libraries consulted**: pydantic-ai Agent instrument Langfuse instrumentation parameter, Pydantic-ai</violation>
</file>
<file name="packages/tracecat-registry/tracecat_registry/integrations/agents/builder.py">
<violation number="1" location="packages/tracecat-registry/tracecat_registry/integrations/agents/builder.py:69">
Enabling global instrumentation at import time alongside per-function `@observe()` can cause duplicate/nested spans and import-time side effects. Consider moving instrumentation to app initialization and using either global or per-function observing, not both.</violation>
<violation number="2" location="packages/tracecat-registry/tracecat_registry/integrations/agents/builder.py:768">
`agent` and `run_agent` are both decorated with `@observe()`, which may produce nested/double traces. Prefer observing only `run_agent` to avoid duplication.</violation>
<violation number="3" location="packages/tracecat-registry/tracecat_registry/integrations/agents/builder.py:888">
The Langfuse integration lacks error handling, making the core agent execution process fragile. Failures in the observability service (e.g., network issues, misconfiguration) could crash the `run_agent` function, preventing agents from running. Calls to the Langfuse client should be wrapped in a `try/except` block to ensure resilience.</violation>
</file>
React with 👍 or 👎 to teach cubic. Mention @cubic-dev-ai to give feedback, ask questions, or re-run the review.
| output_type=response_format, | ||
| model_settings=ModelSettings(**model_settings) if model_settings else None, | ||
| retries=retries, | ||
| instrument=True, # Enable Langfuse instrumentation |
There was a problem hiding this comment.
Instrumentation is always enabled. This can cause unnecessary initialization or errors if Langfuse is not configured. Make it conditional on configured secrets.
DEV MODE: This violation would have been filtered out by GPT-5.
Reasoning:
• GPT-5: instrument=True on pydantic_ai.Agent does not auto-enable Langfuse; without telemetry configuration it’s a no-op. Secrets are optional, so the claimed risk is speculative and low impact.
• Libraries consulted: pydantic-ai Agent instrument Langfuse instrumentation parameter, Pydantic-ai
Prompt for AI agents
Address the following comment on packages/tracecat-registry/tracecat_registry/integrations/pydantic_ai.py at line 334:
<comment>Instrumentation is always enabled. This can cause unnecessary initialization or errors if Langfuse is not configured. Make it conditional on configured secrets.
DEV MODE: This violation would have been filtered out by GPT-5.
Reasoning:
• **GPT-5**: instrument=True on pydantic_ai.Agent does not auto-enable Langfuse; without telemetry configuration it’s a no-op. Secrets are optional, so the claimed risk is speculative and low impact.
• **Libraries consulted**: pydantic-ai Agent instrument Langfuse instrumentation parameter, Pydantic-ai</comment>
<file context>
@@ -312,6 +331,7 @@ def build_agent(
output_type=response_format,
model_settings=ModelSettings(**model_settings) if model_settings else None,
retries=retries,
+ instrument=True, # Enable Langfuse instrumentation
**kwargs,
)
</file context>
[internal] Confidence score: 8/10
[internal] Posted by: General AI Review Agent
| """ | ||
|
|
||
| # Initialize Langfuse client and update trace | ||
| langfuse_client = get_client() |
There was a problem hiding this comment.
The Langfuse integration lacks error handling, making the core agent execution process fragile. Failures in the observability service (e.g., network issues, misconfiguration) could crash the run_agent function, preventing agents from running. Calls to the Langfuse client should be wrapped in a try/except block to ensure resilience.
Prompt for AI agents
Address the following comment on packages/tracecat-registry/tracecat_registry/integrations/agents/builder.py at line 888:
<comment>The Langfuse integration lacks error handling, making the core agent execution process fragile. Failures in the observability service (e.g., network issues, misconfiguration) could crash the `run_agent` function, preventing agents from running. Calls to the Langfuse client should be wrapped in a `try/except` block to ensure resilience.</comment>
<file context>
@@ -876,6 +884,27 @@ async def run_agent(
"""
+ # Initialize Langfuse client and update trace
+ langfuse_client = get_client()
+
+ # Get workflow context for session_id
</file context>
[internal] Confidence score: 9/10
[internal] Posted by: System Design Agent
| secrets=[*PYDANTIC_AI_REGISTRY_SECRETS], | ||
| namespace="ai", | ||
| ) | ||
| @observe() |
There was a problem hiding this comment.
agent and run_agent are both decorated with @observe(), which may produce nested/double traces. Prefer observing only run_agent to avoid duplication.
Prompt for AI agents
Address the following comment on packages/tracecat-registry/tracecat_registry/integrations/agents/builder.py at line 768:
<comment>`agent` and `run_agent` are both decorated with `@observe()`, which may produce nested/double traces. Prefer observing only `run_agent` to avoid duplication.</comment>
<file context>
@@ -759,6 +765,7 @@ class AgentOutput(BaseModel):
secrets=[*PYDANTIC_AI_REGISTRY_SECRETS],
namespace="ai",
)
+@observe()
async def agent(
user_prompt: Annotated[
</file context>
[internal] Confidence score: 8/10
[internal] Posted by: General AI Review Agent
| @observe() | |
| # @observe() |
| from tracecat_registry.integrations.agents.exceptions import AgentRunError | ||
|
|
||
| # Initialize Pydantic AI instrumentation for Langfuse | ||
| Agent.instrument_all() |
There was a problem hiding this comment.
Enabling global instrumentation at import time alongside per-function @observe() can cause duplicate/nested spans and import-time side effects. Consider moving instrumentation to app initialization and using either global or per-function observing, not both.
Prompt for AI agents
Address the following comment on packages/tracecat-registry/tracecat_registry/integrations/agents/builder.py at line 69:
<comment>Enabling global instrumentation at import time alongside per-function `@observe()` can cause duplicate/nested spans and import-time side effects. Consider moving instrumentation to app initialization and using either global or per-function observing, not both.</comment>
<file context>
@@ -63,6 +65,9 @@
from tracecat_registry.integrations.agents.exceptions import AgentRunError
+# Initialize Pydantic AI instrumentation for Langfuse
+Agent.instrument_all()
+
</file context>
[internal] Confidence score: 7/10
[internal] Posted by: General AI Review Agent
|
@cubic-dev-local review this |
@sanxroz I've started the AI code review. It'll take a few minutes to complete. |
|
@cubic-dev-local review this |
@sanxroz I've started the AI code review. It'll take a few minutes to complete. |
|
@cubic-dev-local review this |
@sanxroz I've started the AI code review. It'll take a few minutes to complete. |
There was a problem hiding this comment.
2 issues found across 6 files
Prompt for AI agents (all 2 issues)
Understand the root cause of the following 2 issues and fix them.
<file name="packages/tracecat-registry/tracecat_registry/integrations/agents/builder.py">
<violation number="1" location="packages/tracecat-registry/tracecat_registry/integrations/agents/builder.py:888">
The Langfuse client is initialized and used without checking if the optional Langfuse configuration is present. This can lead to runtime errors when the integration is not configured, as the `langfuse_secret` is defined as optional in `pydantic_ai.py` but used unconditionally in `builder.py`.
DEV MODE: This violation would have been filtered out by GPT-5.
Reasoning:
• **GPT-5**: Filter. Optional Langfuse config and optional trace_id are intentional, and SDK docs don’t indicate exceptions on missing env vars—calls likely no-op. No solid evidence of a runtime error; keeping this would risk a false positive.
• **Libraries consulted**: Langfuse Python SDK get_client observe update_current_trace get_current_trace_id, Langfuse-python, Langfuse-docs</violation>
</file>
<file name="frontend/src/client/types.gen.ts">
<violation number="1" location="frontend/src/client/types.gen.ts:178">
The `trace_id` field was added manually to an auto-generated file. The `AgentOutput` type in `types.gen.ts` and its corresponding schema in `schemas.gen.ts` should be updated by regenerating the client from the backend's OpenAPI schema, not by direct modification.
DEV MODE: This violation would have been filtered out by GPT-5.
Reasoning:
• **GPT-5**: trace_id is present in both auto-generated types.gen.ts and schemas.gen.ts, matching an updated OpenAPI-generated client. Backend adds/returns trace_id; frontend has a generate-client script. No evidence of manual edits; not a valid violation.</violation>
</file>
React with 👍 or 👎 to teach cubic. Mention @cubic-dev-ai to give feedback, ask questions, or re-run the review.
| """ | ||
|
|
||
| # Initialize Langfuse client and update trace | ||
| langfuse_client = get_client() |
There was a problem hiding this comment.
The Langfuse client is initialized and used without checking if the optional Langfuse configuration is present. This can lead to runtime errors when the integration is not configured, as the langfuse_secret is defined as optional in pydantic_ai.py but used unconditionally in builder.py.
DEV MODE: This violation would have been filtered out by GPT-5.
Reasoning:
• GPT-5: Filter. Optional Langfuse config and optional trace_id are intentional, and SDK docs don’t indicate exceptions on missing env vars—calls likely no-op. No solid evidence of a runtime error; keeping this would risk a false positive.
• Libraries consulted: Langfuse Python SDK get_client observe update_current_trace get_current_trace_id, Langfuse-python, Langfuse-docs
Prompt for AI agents
Address the following comment on packages/tracecat-registry/tracecat_registry/integrations/agents/builder.py at line 888:
<comment>The Langfuse client is initialized and used without checking if the optional Langfuse configuration is present. This can lead to runtime errors when the integration is not configured, as the `langfuse_secret` is defined as optional in `pydantic_ai.py` but used unconditionally in `builder.py`.
DEV MODE: This violation would have been filtered out by GPT-5.
Reasoning:
• **GPT-5**: Filter. Optional Langfuse config and optional trace_id are intentional, and SDK docs don’t indicate exceptions on missing env vars—calls likely no-op. No solid evidence of a runtime error; keeping this would risk a false positive.
• **Libraries consulted**: Langfuse Python SDK get_client observe update_current_trace get_current_trace_id, Langfuse-python, Langfuse-docs</comment>
<file context>
@@ -876,6 +884,27 @@ async def run_agent(
"""
+ # Initialize Langfuse client and update trace
+ langfuse_client = get_client()
+
+ # Get workflow context for session_id
</file context>
[internal] Confidence score: 9/10
[internal] Posted by: System Design Agent
| message_history: Array<ModelRequest | ModelResponse> | ||
| duration: number | ||
| usage?: RunUsage | null | ||
| trace_id?: string | null |
There was a problem hiding this comment.
The trace_id field was added manually to an auto-generated file. The AgentOutput type in types.gen.ts and its corresponding schema in schemas.gen.ts should be updated by regenerating the client from the backend's OpenAPI schema, not by direct modification.
DEV MODE: This violation would have been filtered out by GPT-5.
Reasoning:
• GPT-5: trace_id is present in both auto-generated types.gen.ts and schemas.gen.ts, matching an updated OpenAPI-generated client. Backend adds/returns trace_id; frontend has a generate-client script. No evidence of manual edits; not a valid violation.
Prompt for AI agents
Address the following comment on frontend/src/client/types.gen.ts at line 178:
<comment>The `trace_id` field was added manually to an auto-generated file. The `AgentOutput` type in `types.gen.ts` and its corresponding schema in `schemas.gen.ts` should be updated by regenerating the client from the backend's OpenAPI schema, not by direct modification.
DEV MODE: This violation would have been filtered out by GPT-5.
Reasoning:
• **GPT-5**: trace_id is present in both auto-generated types.gen.ts and schemas.gen.ts, matching an updated OpenAPI-generated client. Backend adds/returns trace_id; frontend has a generate-client script. No evidence of manual edits; not a valid violation.</comment>
<file context>
@@ -175,6 +175,7 @@ export type AgentOutput = {
message_history: Array<ModelRequest | ModelResponse>
duration: number
usage?: RunUsage | null
+ trace_id?: string | null
}
</file context>
[internal] Confidence score: 10/10
[internal] Posted by: System Design Agent
Summary by cubic
Integrates Langfuse observability into agent runs, capturing traces and returning a trace_id with agent results. Adds optional Langfuse config and updates the frontend client types to surface the trace_id.
New Features
Dependencies