Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parameter Extractor Node Costs Not Visible in Langfuse #14272

Open
5 tasks done
lyestarzalt opened this issue Feb 24, 2025 · 2 comments
Open
5 tasks done

Parameter Extractor Node Costs Not Visible in Langfuse #14272

lyestarzalt opened this issue Feb 24, 2025 · 2 comments
Labels
🐞 bug Something isn't working

Comments

@lyestarzalt
Copy link

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

0.15.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

  1. Create a workflow with a Parameter Extractor node
Image
  1. Configure Langfuse integration
  2. Run the workflow
  3. Check Langfuse dashboard for cost information
  4. LLM node costs are visible, but Parameter Extractor costs are not shown

✔️ Expected Behavior

Langfuse should display cost information for Parameter Extractor nodes.

❌ Actual Behavior

Cost information is only visible for LLM nodes in Langfuse, but not for Parameter Extractor nodes.

Image
Copy link

dosubot bot commented Feb 24, 2025

Hey @lyestarzalt, I'm here to help you with any bugs, questions, or contributions you have for Langgenius' Dify project. Let's tackle this issue together!

The issue you're experiencing, where cost information for Parameter Extractor nodes is not visible in Langfuse, might be related to how the metadata is being sent. According to the Dify codebase, the metadata from the Parameter Extractor node does include cost information, such as total tokens, total price, and currency [1].

Here are a few things you might want to check:

  1. Ensure Metadata is Sent Correctly: Verify that the metadata containing cost information is being correctly sent from the Parameter Extractor node. The relevant metadata keys are NodeRunMetadataKey.TOTAL_TOKENS, NodeRunMetadataKey.TOTAL_PRICE, and NodeRunMetadataKey.CURRENCY.

  2. Langfuse Configuration: Double-check the Langfuse configuration to ensure that it is set up to display cost information for all node types, not just LLM nodes.

  3. Logging and Error Handling: Look for any logging or error messages in the Dify logs that might indicate issues with sending or processing the cost data for Parameter Extractor nodes.

If these checks do not resolve the issue, it might be helpful to review the specific implementation details in your setup or consult any additional documentation or support resources available for Dify and Langfuse.

To continue talking to Dosu, mention @dosu.


Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot dosubot bot added the 🐞 bug Something isn't working label Feb 24, 2025
@lyestarzalt
Copy link
Author

From Langfuse

LLM node:

id name model startTime totalTokens inputPrice outputPrice calculatedInputCost calculatedOutputCost
654--ef6 llm gpt-4 2025-02-24T12:54:35.332Z 25 0.00003 0.00006 0.00069 0.00612 (LLM Node)
5f894395-.. llm gpt-4 2025-02-24T12:54:23.262Z 25 0.00003 0.00006 0.00069 0.00612 (LLM Node)

Parameter Extractor Node:

id name model startTime totalTokens inputPrice outputPrice calculatedInputCost calculatedOutputCost
b186f4..c669780 llm null 2025-02-24T12:52:21.666Z 658 null null null null (Parameter Extractor)
61d6..3dd11 llm null 2025-02-24T12:51:10.435Z 655 null null null null (Parameter Extractor)
2320..0e5736 llm null 2025-02-24T12:50:27.542Z 633 null null null null (Parameter Extractor)

From Dify

The Parameter Extractor trace JSON shows:

{
  "usage": {
    "prompt_tokens": 623,
    "prompt_unit_price": "0.03",
    "prompt_price_unit": "0.001",
    "prompt_price": "0.0186900",
    "completion_tokens": 8, 
    "completion_unit_price": "0.06",
    "completion_price_unit": "0.001",
    "completion_price": "0.0004800",
    "total_tokens": 631,
    "total_price": "0.0191700",
    "currency": "USD",
    "latency": 1.4745679500047117
  }
}

In Dify trace logs, Parameter Extractor nodes correctly calculate and track token usage and costs, but these values aren't properly transmitted to Langfuse's cost tracking fields, while LLM nodes successfully populate these same fields.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐞 bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant